I have run into a situation where my software is attempting to compress decompress a pretty large file using TZipFile and seems to be choking on it. In the case of decompress, the compressed size is almost 1G and the decompressed file size is over 14G. When decompressing it finds the file but is unable to "ExtractAll" giving the error "item not found". It looks like it is failing in System.Zip.TZipFile.Read.

Likewise, when trying to compress the 14G file, it seems to be running fine and it creates the compressed file and completes without error. However, I know that when successfully compressed, the file should be almost 1GB. The resulting file is a fraction of that and when an attempt is made to decompress it, it turns out to be corrupt as one would expect based on my previous observations.

So is there a size limitation for the TZipFIle object and (if so) what are those limitations?

On 11/20/2014 3:15 PM, Rod Sherer wrote:
> Here was an odd one that I encountered though.
> Deleting that file out of the compressed file must have done
> something wonky that messed up TZipFile.

The Zip64 extensions are optional on a per-file basis, and can be used
for smaller file sizes when they're not actually necessary.  It's also
possible to add or remove files to a zip without updating all of the
other compressed files' headers.  What more than likely happened was
that the original zip used the extensions and WinRAR didn't remove them
when it deleted the file.  The archive was still well formed, just still
not compatible with older parsers.

-- 
Craig Peterson
Scooter Software

On 11/20/2014 2:13 PM, Rod Sherer wrote:
> You would think they would have a 64 bit version of TZipFile since I am building a 64 bit application. Oh well.

TZipHeader is basically the record exactly as it's written in the zip
file itself, so it can't depend on your application being 32-bit or
64-bit. Support for larger files was added to the format by storing the
larger file size in the ExtraField portion of the header, but that's a
user-extensible data structure (backwards compatible Unicode filenames
are stored there too), so it's not as easy to parse.

-- 
Craig Peterson
Scooter Software

Thanks. I was just running tests to verify the 4G limit and my tests with files over 2G were failing. Anything under 2G was fine. 

Here was an odd one that I encountered though. I wanted to see if maybe you could have two files which individually weren't over 2 G, but together they were. I used windows to create my compressed folder (zip file) and then tried using my application to decompress it. It failed. I deleted one of the files from the compressed folder (using WinRAR because I really wasn't thinking about it when I did it) and it still failed. I was able to decompress the file through other means. It wasn't corrupt. And I was 
able to create a new zip file with the same file in it, and TZipFile worked fine with that. Deleting that file out of the compressed file must have done something wonky that messed up TZipFile. Just thought that was odd.

Thanks again.

On 11/20/2014 1:50 PM, Rod Sherer wrote:
> So is there a size limitation for the TZipFIle object and (if so) what are those limitations?

System.Zip does not support the Zip64 extensions to handle files larger
than 2GB.  For that you'll need to use a different library, such as
Abbrevia[1], ZipForge[2], or ZipTV[3].

1: http://www.sourceforge.net/projects/tpabbrevia
2: http://www.componentace.com
3: http://www.ziptv.com

System.Zip is limited to 2GB for both the archive itself and the
compressed and uncompressed sizes of any stored files, and at most 65K
compressed files.

-- 
Craig Peterson
Scooter Software

Thanks. That seems to be the bit of information I was missing.

You would think they would have a 64 bit version of TZipFile since I am building a 64 bit application. Oh well.

Thanks again.

> {quote:title=Rod Sherer wrote:}{quote}
> I have run into a situation where my software is attempting to compress decompress a pretty large file using TZipFile and seems to be choking on it. In the case of decompress, the compressed size is almost 1G and the decompressed file size is over 14G. When decompressing it finds the file but is unable to "ExtractAll" giving the error "item not found". It looks like it is failing in System.Zip.TZipFile.Read.
> 
> Likewise, when trying to compress the 14G file, it seems to be running fine and it creates the compressed file and completes without error. However, I know that when successfully compressed, the file should be almost 1GB. The resulting file is a fraction of that and when an attempt is made to decompress it, it turns out to be corrupt as one would expect based on my previous observations.
> 
> So is there a size limitation for the TZipFIle object and (if so) what are those limitations?

4Gig file limit at a guess 

http://en.wikipedia.org/wiki/Zip_(file_format)

and what do you see in System.zip but
TZipHeader = packed record
    MadeByVersion:      UInt16; // Start of Central Header
    RequiredVersion:    UInt16; // Start of Local Header
    Flag:               UInt16;
    CompressionMethod:  UInt16;
    ModifiedDateTime:   UInt32;
    CRC32:              UInt32;
    CompressedSize:     UInt32;
    UncompressedSize:   UInt32; //<------------------------------- 4G
    FileNameLength:     UInt16;
    ExtraFieldLength:   UInt16; // End of Local Header
    FileCommentLength:  UInt16;
    DiskNumberStart:    UInt16;
    InternalAttributes: UInt16;
    ExternalAttributes: UInt32;
    LocalHeaderOffset:  UInt32; // End of Central Header
    FileName: TBytes;
    ExtraField: TBytes;
    FileComment: TBytes;
  end;

Make you file into parts 4 and zip those

--
Linden
"Mango" was Cool but "Wasabi" was Hotter but remember it's all in the "source"