7zip, lzma, LZ4, fastLZ, zip/gzip & brotli native multiplatform plugins.

@elias_t
How to package multiple files into a single file using FastLZ

 

Hello.

FastLZ, Brotli and LZ4 pack single files only.

For multiple files you should create your own file container.

However in the next update tar packing is coming.
Also I have planned to create a custom file container to support various compression formats.

 

Hey there - I have a little problem.

I have a folder that contains some localization files with non-standard characters in their filenames.
7-zip works fine, but with your plugin the filenames get broken.


Left: 7-zip - Right: Your Plugin

 

Hello.

Please send me a pm with a sample 7z archive that replicates the issue.
Also, on which OS are you testing and plugin version.
Thanks.

Edit: Since version 2.6.2 unicode issues on windows should be resolved. I just ran some tests on win7/win10 with 7zip created archives that contain unicode folders and files and it was running as it should.

 

Yeah it happens on Win10 x64, I'll check the plugin version again and send you a sample if it still happens on the latest version.

EDIT: Works now on the latest version

 

Cool!

Thx for the follow up.

 

Yeah, the update just didn't work at first because Visual Studio seems to lock DLLs. I can't update any plugins without closing VS first.

 

version 2.7 [Major update]

- 7ZIP: Added byte level progress for 7zip extraction (whole files/fileBuffers or entries)
- 7ZIP: Added byte level progress for lzma alone creation/decompression.
- 7ZIP: For this reason 3 new functions have been introduced. See examples and docs.
- 7ZIP: Improved entry search and extraction speed.
- 7ZIP: Fixed an issue with entries extraction, where some unwanted empty folders could be extracted.
- 7ZIP: Decompress buffer imporoved.
- 7ZIP: Fixed issues with files larger then 2GB on 64bit systems.
- 7ZIP: Linux: reintroduced FileBuffers for Linux.
- 7ZIP: Protection against illegal characters in 7z entries.
- 7ZIP: The get7zInfo and get7zSize functions run faster and no more a temp log is needed.

- LZ4: Progress of decompression is a ulong now.


- ZIP: Added support for tar/untar.
- ZIP: Added support for tar.gz, tar.bz2 creation.
- ZIP: Protection against illegal characters in entries.
- ZIP: Added ability to create and decompress bz2 files.
- ZIP: Faster reaction to the cancel command.
- ZIP: Fixed gzip file decompression issues.
- ZIP: gzip uncompress progress relies on the compressed files size now.

- ALL: tvOS uses the same functions as WebGL now, since it does not allow access to the file system.
- ALL: Combined tvOS and WebGL in one demo scene.
(WebGL sample scenes renamed to reflect the above changes.)
- ALL: Updated and refactored the demo scene and scripts to reflect the new changes.
- ALL: Removed the Streaming Assets folder files. All example files are downloaded now.

- ALL:LINUX: Only one glibc version of Linux plugins now provided (glibc2.17)

In general, a lot more has been done to improve the stability and usability of the plugins.

 

Hello.

I am looking for someone who is willing to test the zip plugin for Nintendo Switch or PS4 with a clang5.0/llvm compiled version.

 

Hello @elias_t -
I have purchased your plugin and find it to be very fast and useful as you describe. Given that it is a compiled plugin, would it be possible for you to supply Windows ARM 64 binaries so that your plugin will work with the Hololens 2?
Many thanks.

 

Hello.

UWP was part of the package in the past, but I have discontinued it because there were many issues with the Microsoft Store certification process.
However I could provide you the Windows arm64 plugin.
Later today I will send you in a pm the binary.

 

Hi elias_t,

I have been trying to get the number from progress[0] while decompressing zip file, but, it stayed at zero, until it is done. I used 2 coroutines, but it didn't work.

Do you have any example on how to use progress[0]? I tried the example that came with the library, it doesn't work.

Thanks.

 

Hello.

You should be using a separate thread to call your function.
Also take note that there are 2 progress variables: the int[0] that indicates the files decompressed progress and the ulong[0] that reports the total bytes that have been decompressed.

If you still have issues please send me a private message with your code so I can check.

 

Hi, I tried to decompress a zip without freezing the main thread but I think I'm doing something wrong because it doesn't work and still freezes my app. (building for android) I read that it should be possible to run it in a background thread so can somebody point out what I'm doing wrong. thanks in advance

Code (CSharp):

1. IEnumerator unpackZip(){
2.        
3.         string file = GetFolderPath() + "/" + toDownload[0].zipName;
4.         zPass = zPassT.text;
5.         int[] prog = new int[1];
6.         ulong[] byteProgress = new ulong[1];
7.  
8.         lzip.decompress_File(file, GetFolderPath() + "/", prog,null, byteProgress, toDownload[0].zipPassword);
9.         yield return null;
10.        
11.         debugText.text += "\n" + toDownload[0].zipName + " deleted";
12.         DeleteFile(toDownload[0].zipName);
13.         toDownload.Remove(toDownload[0]);
14.         UpdateQueue();
15.         if (toDownload.Count > 0){
16.             string url = dPath + toDownload[0].zipName;
17.             string folderPath = GetFolderPath();
18.             StartCoroutine(DownloadFile(url, folderPath, toDownload[0].zipName));
19.         }
20.         yield return null;
21.     }

 

Hello.
You should call this in a separate thread.
The demo script has an example.

 

What script do you mean with the demo script? I can't find a script named demo.

 

Hello.

SevenZipTest.cs

 

update version 2.7.1

- ZIP: Fixed Unicode issues with the windows plugins.

 

update version 2.7.2 [important update]

- ZIP: Windows: Resolved all issues with Unicode paths (reading and writing)
- ZIP: All platforms: All file paths/names will be treated as utf-8
- ZIP: Gzip file decompression got 4 times faster.
- FLZ: Fixed the decompress buffer function.

 

update version 2.7.3

- ZIP: Windows: reintroduced the lzip.setEncoding function, since some applications do not use utf8 encoding for filenames.

 

Now on sale at 30% off.
After this there will be a $5 price increase.

 

Hi Elias_t,

I love your Plug-ins, it is so fast. Now, I am about to publish my app as Desktop app without going thru' MS Store, the instruction kept pointing to building the app in UWP mode. I tried using PC x86 and x64, I kept getting 178 errors as shown in the screenshot. Do you have any suggestions what to do with these "does not exist in the current context" errors?



I really appreciate all your help and your time.

 

Hello.

Do you have the uwp plugins? If not, then this is the case.
I am sending you a pm.

 

Update version 2.7.4

- Brotli: Added support for WebGL to allow decompression of brotli buffers.
- 7Zip: Fixed some issues with the demo script.
- ZIP: Fixed some preprocessor issues that caused compilation errors on some platforms.
- ZIP: The setPermissions function was accidentaly removed in a previous update. Now reinstated.
- ZIP: Fixed date issues when extracting zip archives on unix operating systems.

 

Hi elias_t,
This is a good plug-in, but I have encountered some problems with memory. I have some relatively large files. Decompression on the phone will consume a lot of memory. Is there a way to support streaming decompression? For example, there is only a fixed-size cache area. Each time it is decompressed into the cache area, then the content of the cache area is directly written to a local file, and the loop is repeated until the file is decompressed.
I use sevenZip. @elias_t

 

Hello. Are you referring to the 7zip plugin? If yes, then use the large files flag. If not please send me a private message with more information.

 

Yes it is, I'll try the api with the large files flag. i want to know how much memory is applied for in this way, or what is the implementation method, I want to strictly control the amount of memory used on the mobile phone.

 

If you use the large files flag you should be ok.

 

Thanks, i‘ll try.

 

Update version 2.7.5

- Zip: Fix for Marshal.StringToCoTaskMemAuto -> Marshal.StringToHGlobalAuto on Unity 2019.3 or older (il2cpp Android)
- Brotli: buffer functions got uncompressed size parameter.
- MacOS bundles: BigSur fix. Optional silicon support via zipped .bundles (MacOS 10.9 minimum)

 

Update version 2.8

- ZIP: Critical Fix for the delete_entry functions.
- ZIP: Added MacOS bundles: Intel only, Silicon only, Universal (Intel + Silicon)

(The plugin is uploaded now with Unity versions: 2018, 2019, 2020, 2021. If anyone needs it for 2017 or Unity 5.6 please message me with your invoice number.)

 

Hello,

I have tested 7Zip and Zip decompression on windows computer and android phones and all seems to work pretty well, this is a great value to add to your projects!

________________

Just a little suggestion: the documentation is somewhat unclear regarding ZIP decompression with a splitled archive (archive.zip, archive.z01, archivz.z02 ...)

Code (CSharp):

1. lzip.decompress_File(FilePath, ExtractionPath, progress, null, progress2);

seems to work perfecly fine with a simple or a splitled archive, all we need to do is to send the name of the main archive file (my_archive.zip) as the FilePath and the function will automatically load any additional splitled archive (my_archive.z01 ...).

This was not so obvious to me and I was looking for a function with a string array or a string List as an input.
I may lack some knowledge regarding compression and decompression but that is beside the point.

________________

I suggest updating Instructions_ZIP.pdf, especially regarding the extractions methods like decompress_File and mention that splitled archives are indeed supported out of the box without any additional arguments/parameters.

Thank you and have a great day!

Sorry for my somewhat poor english level.

 

Thank you for your input!

Since the split archives are a supported feature I have not added this extra info in the function.
i will add this info on the next update.

Thank you.

 

Update version 2.8.1

- Added Android x86_64 plugins to support Chrome OS (2020 and 2021 versions).

 

support uwp arm？

 

Hello.

It is unofficially supported.
That means I can send you the plugins but they will not certify for Microsoft store.

 

Update version 2.9

- Updated WebGL plugins for Unity 2021+
- Added a custom download function to the plugins, to download a file to a native (unmanaged) memory buffer.
- All Plugins: Ability to pass an IntPtr unmanaged buffer as a file buffer where supported.
- ZIP: IntPtr (native memory buffers) and inMemory lzip class can be passed now as a file Buffer.
This allows avoiding memory spikes when downloading large files to unzip.
- ZIP: Gzip file functions return -8 now on cancel.

- Brotli:WebGL: fixed an issue with not releasing a buffer on decompression.
- Brotli:WebGL: fixed an issue with Development builds.

- Native buffer examples updated because dropbox would not allow the webrequest function to work properly.
- ZIP: CompressDir function: fixed an issue where the inplace zip creation (without adding a custom output file) would truncate the last character.
- Removed Linux x86 plugins for Unity 2019+ (no longer supported by Unity)
- Added Android x86_64 plugins for Unity 2019+
- Removed Windows x86 plugins for Unity 2021+ (no longer supported by Unity)

 

Hi elias_t!

I want to replace my LZ4 plugin with this one, but it seems not compatible with other LZ4 plugins.
I'm using lz4 to communicate with server, so I should compress some byte arrays in memory. but compressBuffer() and decompressBuffer() method does not work with our server.
I tried to compare your results with my results from python lz4.frame package, and I found some headers and footers are missing.

ADL.LZ4 plugin

b'\xf1-Lorem Ipsum is simply dummy text of the printing and typeset\x10\x00\xa8industry. K\x00\x81has been<\x00\x04#\x00\xb8\'s standardb\x00\xa1ever since.\x00\xf2\x071500s, when an unknown\x81\x00\xf1\x01er took a galley\x9e\x001ype\x96\x00\xf2\x07scrambled it to make a \x00\xf1\x02specimen book. It\xa2\x00\xf1\x1bsurvived not only five centuries, but also\x9a\x00\xf7\x06leap into electronic \x0c\x01\x80, remain\x17\x01\xf1\x06essentially unchangedy\x00\xd0was popularis\xad\x00\x11n]\x00\xa11960s with\x0f\x00prelease\xe1\x00\xf3\x05Letraset sheets contf\x00\x08s\x01`passag\xb5\x00\x00\r\x010morE\x00\x00\xcb\x00"lyW\x00\xf0\x00desktop publish@\x00\xf0\x15software like Aldus PageMaker includ(\x00\x81versions\x8f\x00\xa0orem Ipsum'

lz4.frame package

b'\x04"M\x18h@=\x02\x00\x00\x00\x00\x00\x00\x17\xf7\x01\x00\x00\xf1-Lorem Ipsum is simply dummy text of the printing and typeset\x10\x00\xa8industry. K\x00\x81has been<\x00\x04#\x00\xb8\'s standardb\x00\xa1ever since.\x00\xf2\x071500s, when an unknown\x81\x00\xf1\x01er took a galley\x9e\x001ype\x96\x00\xf2\x07scrambled it to make a \x00\xf1\x02specimen book. It\xa2\x00\xf1\x1bsurvived not only five centuries, but also\x9a\x00\xf1\x05leap into electronic_\x00\x03\x0c\x01\x80, remain\'\x01\xf1\x06essentially unchangedy\x00\xd0was popularis\xad\x00\x11n]\x00\xa11960s with\x0f\x00prelease\xe1\x00\xf3\x05Letraset sheets contf\x00\x08s\x01`passag\xb5\x00\x00\xa3\x010morE\x00\x00\xcb\x00"lyW\x00\xf0\x00desktop publish\xa6\x00\xf0\x15software like Aldus PageMaker includ(\x00\x81versions\x8f\x00\xa0orem Ipsum\x00\x00\x00\x00'

Would you help me with this?

 

Hello. lz4.frame uses its own method to write and read headers and footers.
You can easily modify the LZ4.compressBuffer and LZ4.decompressBuffer functions to comply with the lz4.frame method.

This page has the info: https://python-lz4.readthedocs.io/en/stable/lz4.frame.html

If you find difficulties please contact me again in a private message and send me the 2 binaries that you posted above.
Regards.

 

update version 2.9.1

- Fixed compatibility issues with Unity 2022 for the Lz4, Lzma, Brotli and Flz plugin scripts.
- ZIP: Updated the WebGL demo script to work correctly with NativeFileBuffers. (On Unity2022.x+ you should enable http connection for the demo)
- Brotli: Added a separate WebGL demo scene.

 

Update version 2.9.2

- ZIP: Added the extract_entries function which allows to extract a specific list of entries of a zip archive.

 

How to compress the folder and decompress, just like the compression software function, I only see a simple compression of a single file demo, and not compress a folder and then decompress, can you do it? I need this feature.

 

Hello. Please specify for which plugin you are talking about.

 

I need to unzip some files into an UWP project. Is there some alternative to this plugin?

 

Hello.

UWP is unofficially supported. I can provide binaries for UWP but they will not pass Microsoft Store certification.

 

Well, I just need to unzip some downloaded .zip files. I don't need to zip nor use any other function. Do you think it would still be a problem to the MS Store?

 

Hello, I saw the overview says "compress/decompress buffers to/from zlib/gzip streams", but can't find the API which support streams.

Because I' m using the BetterStreamingAssets plugin, that help's me extract bundles directly from StreamingAssets to hard disk to skip reading and decompress them to memory, which can greatly improve decompression performance.

Code (CSharp):

1.                     //var ms = new MemoryStream(job.bytes);
2.                     var ms = BetterStreamingAssets.OpenRead($"assetBundle/{job.assetInfo.path}.sta");
3.                     var fs = File.Create(outputPath);
4.  
5.                     // want to replace this line by using native plugin
6.                     GZipCompresser.uncompress(ms, fs);
7.  
8.                     ms.Dispose();
9.                     fs.Dispose();
10.  

 

Hello and sorry for the late reply. I was away.

The function you are looking for is lzip.ungzipFile

 

Thanks, but I can only find this version of ungzipFile function, which can not pass the parameter type of stream.
The BetterStreamingAssets plugin, different from the System.IO.File class, it uses its internal special read method to read bytes from streamAssets on andriod and ios and returns a stream to user.

 

Ah sorry. I misunderstood. This function does not exist. I will look if I can add one in the next update.