Hi all,

Just a quick query - which is better, in terms of speed and memory 
usage: a static or a dynamic array.

For example, consider

  data: array[0..819200] of Byte;

where we might only use 9760 bytes on some occasions, but may use the 
full 819200 on others, or:

  data: array of Byte;

and then SetLength(data,9760) or SetLength(data,819200)...or any other 
size, depending on the size of the file being loaded. I know these days, 
800KB in nothing (when most PCs have in excess of 1GB of RAM).

Cheers,

Gerald.

On 21/01/2015 20:01, Tom Brunberg wrote:
> Gerald Holdsworth wrote:


>> Here is what I've found:
>>
>> * With 'data' as a global, public, dynamic array, it gets corrupted;
>> * if I change it back to a static array (and obviously remove the
>> SetLength call), it all works fine;
>> * if I move it into the procedure, so it is now local, but dynamic, and
>> call the second procedure with 'data' as a parameter (as you suggest
>> Tom, and as I've tried before, but with the array as a global dynamic),
>> it works.
>
> Gerald,
> What you say above makes no sense at all. The *only* possible difference in accessing static arrays
> versus dynamic arrays, is that dynamic arrays always have a zero based index (first element has
> index 0), while static arrays are indexed according the type declaration (e.g. array[1..77]. Since
> you in earlier post talked about array[0..819200] this is not the reason.
>
> You are definitely doing something wrong.

I've been over the code, many times - I can't see what might be causing 
it. However, with that said, I have noticed before that declaring a 
static array then going past it's boundary never caused an error, but 
did cause another array to become corrupt - that took a lot of looking 
for. So, you can see why I'm wanting to ditch the static arrays.

> For example, after you read the data, you analyse it, how?

Just reads data - doesn't do any writing back to the array.

> What exactly goes wrong in that anaysis that leads to rejection?

The app loads data files, which were used by retro software, and 
displays the graphic representation on the screen. When the array gets 
corrupted, it becomes obvious on the screen. I had thought of saving 
this bitmap (non-corrupted and corrupted) and posting it here (well, a 
link to it here), but then I got around the issue.

>
> I or anybody else do not see your screen from where we are. Guessing and speculating is a waste of
> time.

Yep - but you, and everyone else on here, have helped. And I thank you 
for that.

Cheers,

Gerald.

Gerald Holdsworth wrote:

> On 20/01/2015 16:12, Tom Brunberg wrote:
> > Gerald Holdsworth wrote:
> > 
> 
> > 
> > Gerald,
> > I can assure you, with 100% certainty, that the data in the array doesn't change by itself. I
> > suggest you step through the analysing part with F7/F8 a line at the time, verifying that it
> > does what it is supposed to do.
> 
> > 
> > Is there any other reason for the copying apart from the name issue (which isn't an issue at
> > all).  If you ditch the local variable named 'data' and instead pass the array from the
> > MainForm as a parameter to the procedure, like you did above with the disc images, you don't
> > need to copy it. And of course the procedure declares the array parameter with the name 'data'
> > here too.
> 
> Here is what I've found:
> 
> * With 'data' as a global, public, dynamic array, it gets corrupted;
> * if I change it back to a static array (and obviously remove the 
> SetLength call), it all works fine;
> * if I move it into the procedure, so it is now local, but dynamic, and 
> call the second procedure with 'data' as a parameter (as you suggest 
> Tom, and as I've tried before, but with the array as a global dynamic), 
> it works.

Gerald,
What you say above makes no sense at all. The *only* possible difference in accessing static arrays
versus dynamic arrays, is that dynamic arrays always have a zero based index (first element has
index 0), while static arrays are indexed according the type declaration (e.g. array[1..77]. Since
you in earlier post talked about array[0..819200] this is not the reason.

You are definitely doing something wrong.

> 
> Therefore, as a global dynamic array it gets corrupted, but as either a 
> static global, or a local dynamic, array it is all fine. Could this to 
> do with a previous poster who pointed out where the information is 
> stored (heap/stack) depending on whether it is a 
> static/dynamic/local/global array?

Only if you would run out of stack space, in which case you would get an error, or the program
would simply crash (stop running).

> And finally, what I did notice, is that the size of the global dynamic 
> array differed from what I set it to - e.g. I did a 
> SetLength(data,134508) and it was sized to 131127 elements (can't 
> remember the exact values).

You must be looking at different arrays or you have no clue what your program is doing. As I said
before, things do not change by them self or by some woodoo.
From where did you get the number you used in SetLength? How did you determine that the size was
different?

You really need to show the code that touches the data.

For example, after you read the data, you analyse it, how?
What exactly goes wrong in that anaysis that leads to rejection?
 
I or anybody else do not see your screen from where we are. Guessing and speculating is a waste of
time.

Cheers

-- 
Tom Brunberg
firstname.lastname@welho.com

On 20/01/2015 16:12, Tom Brunberg wrote:
> Gerald Holdsworth wrote:
>

>
> Gerald,
> I can assure you, with 100% certainty, that the data in the array doesn't change by itself. I
> suggest you step through the analysing part with F7/F8 a line at the time, verifying that it does
> what it is supposed to do.

>
> Is there any other reason for the copying apart from the name issue (which isn't an issue at all).
> If you ditch the local variable named 'data' and instead pass the array from the MainForm as a
> parameter to the procedure, like you did above with the disc images, you don't need to copy it. And
> of course the procedure declares the array parameter with the name 'data' here too.

Here is what I've found:

* With 'data' as a global, public, dynamic array, it gets corrupted;
* if I change it back to a static array (and obviously remove the 
SetLength call), it all works fine;
* if I move it into the procedure, so it is now local, but dynamic, and 
call the second procedure with 'data' as a parameter (as you suggest 
Tom, and as I've tried before, but with the array as a global dynamic), 
it works.

Therefore, as a global dynamic array it gets corrupted, but as either a 
static global, or a local dynamic, array it is all fine. Could this to 
do with a previous poster who pointed out where the information is 
stored (heap/stack) depending on whether it is a 
static/dynamic/local/global array?

Also, even though the array is now local, do I still need to do a 
'data:=Nil;' at the end of the function - I've actually just added it, 
prior to making it local, to tidy things up. I'm assuming I can now 
safely remove it as closing the procedure will free it up anyway.

And finally, what I did notice, is that the size of the global dynamic 
array differed from what I set it to - e.g. I did a 
SetLength(data,134508) and it was sized to 131127 elements (can't 
remember the exact values).

Well, it looks like we got there, but raised some fresh questions. Thank 
you Jeff and Tom.

Cheers,

Gerald.

Gerald Holdsworth wrote:

> On 19/01/2015 20:39, Tom Brunberg wrote:
> > > 
> 
> > 
> > We first need to know exactly what the problem is. Then I'm sure we can find a cure.
> 
> Yep, it's a difficult one to explain. I can stop the execution after the 
> file is read in, and 'hover' over the array and it looks fine, but when 
> it is analysed, the incorrect information is given meaning that the, 
> otherwise OK, file is rejected.

Gerald,
I can assure you, with 100% certainty, that the data in the array doesn't change by itself. I
suggest you step through the analysing part with F7/F8 a line at the time, verifying that it does
what it is supposed to do.

> After the file is read into the array, it may be passed to another 
> procedure, depending on it's extension (what that is, is not important). 
> I have three functions, for reading in old retro disc images of 
> different formats. One of the declarations is:
> 
> function readDFSDisc(var data: array of Byte;format: String): TDisc;
> 
> I call this function by disc:=readDFSDisc(data,'.ssd');

The above is correct.

> Should this be, in this case, disc:=readDFSDisc(data[0],'.ssd');  ?
> (and the same for the other, similar, functions)
> Incidentally, 'data' does not get altered in any way within these functions.
> 
> However, there are other circumstances where the file loaded is not a 
> retro disc image, but an actual retro data file. This is not accessed by 
> this procedure, but by another in another unit (hence why 'data' is 
> public). The first thing this other procedure does is to copy the 
> information from one array to another (local to the procedure, but also 
> dynamic...but static also has the same effect):
> 
>     for x:=0 to MainForm.fsize-1 do
>      data[x]:=MainForm.data[x];
> 
> (this is easier than changing the huge number of occurrences of the 
> local 'data' to MainForm.data...and there is probably an easier way of 
> copying said array - Copy method perhaps?)

Is there any other reason for the copying apart from the name issue (which isn't an issue at all).
If you ditch the local variable named 'data' and instead pass the array from the MainForm as a
parameter to the procedure, like you did above with the disc images, you don't need to copy it. And
of course the procedure declares the array parameter with the name 'data' here too.

> As I said, it is difficult to explain without actually showing you in 
> person. However, if I have a brainwave on a better way to illustrate it, 
> I'll do it.

Actual code explains more than a thousand word (hmmm ... I've heard that before ...).

Regards
Tom

-- 
Tom Brunberg
firstname.lastname@welho.com

On 19/01/2015 20:39, Tom Brunberg wrote:
>>

>
> We first need to know exactly what the problem is. Then I'm sure we can find a cure.

Yep, it's a difficult one to explain. I can stop the execution after the 
file is read in, and 'hover' over the array and it looks fine, but when 
it is analysed, the incorrect information is given meaning that the, 
otherwise OK, file is rejected.

After the file is read into the array, it may be passed to another 
procedure, depending on it's extension (what that is, is not important). 
I have three functions, for reading in old retro disc images of 
different formats. One of the declarations is:

function readDFSDisc(var data: array of Byte;format: String): TDisc;

I call this function by disc:=readDFSDisc(data,'.ssd');
Should this be, in this case, disc:=readDFSDisc(data[0],'.ssd');  ?
(and the same for the other, similar, functions)
Incidentally, 'data' does not get altered in any way within these functions.

However, there are other circumstances where the file loaded is not a 
retro disc image, but an actual retro data file. This is not accessed by 
this procedure, but by another in another unit (hence why 'data' is 
public). The first thing this other procedure does is to copy the 
information from one array to another (local to the procedure, but also 
dynamic...but static also has the same effect):

    for x:=0 to MainForm.fsize-1 do
     data[x]:=MainForm.data[x];

(this is easier than changing the huge number of occurrences of the 
local 'data' to MainForm.data...and there is probably an easier way of 
copying said array - Copy method perhaps?)

As I said, it is difficult to explain without actually showing you in 
person. However, if I have a brainwave on a better way to illustrate it, 
I'll do it.

Cheers,

Gerald.

Jeff Overcash (TeamB) wrote:

> > var
> >   data: array of Byte;
> >   F: File;
> >   fsize, amt,ptr,file_size,l: Integer;
> >   file_buffer: array[0..$FFF] of Byte;
> > begin
> >   AssignFile(F,filename);
> >   Reset(F,1);
> >   fsize:=FileSize(F);
> >   file_size:=fsize;
> >   SetLength(data,fsize);
> >   ptr:=0;
> >   while file_size>0 do
> >   begin
> >    BlockRead(F,file_buffer,Length(file_buffer),amt);
> >    for l:=0 to amt-1 do
> >     data[l+ptr]:=file_buffer[l];
> >    file_size:=file_size-amt;
> >    ptr:=ptr+amt;
> >   end;
> >   CloseFile(F);
> >   AssignFile(F,filename+'test.dat');
> >   ReWrite(F,1);
> >   BlockWrite(F,data,fsize,amt);
> >   CloseFile(F);
> > end;
> 
> 
> two bugs with this line.
> 
> 1. Should be data[0].  You want to pass the start of the array, not the address 
> of the variable that points to the start of the array.
> 2. fsize is <= 0 at this point.  You decrement it in your while loop

Jeff, actually fsize has the correct value. He is using another variable, file_size as a counter.

> Changing it to
> 
>     BlockWrite(F, data[0], Length(data), amt);
> 
> worked for me.

fsize can be used here.

Cheers

-- 
Tom Brunberg
firstname.lastname@welho.com

Gerald Holdsworth wrote:
> On 19/01/2015 15:02, Tom Brunberg wrote:
>> Gerald Holdsworth wrote:
>>
> 
>> Please show the procedure(s) where you load original file, create a new one (file or array?) and
>> save it to disk.
> 
> This has been concatenated for clarity. 'data' and 'fsize' are declared 
> in the public section of the Form declaration, while 'F' and 'amt' are 
> declared locally in the procedure. Also, the code is on the other 
> computer, which is currently turned off. If I change it to how Jeff 
> suggested (which is how the code now currently looks) it still works the 
> same.
> The code to save the data is only temporary, and if I access the array 
> 'data' later on (which I do), I still get the corrupted data.
> 
> var
>   data: array of Byte;
>   F: File;
>   fsize, amt,ptr,file_size,l: Integer;
>   file_buffer: array[0..$FFF] of Byte;
> begin
>   AssignFile(F,filename);
>   Reset(F,1);
>   fsize:=FileSize(F);
>   file_size:=fsize;
>   SetLength(data,fsize);
>   ptr:=0;
>   while file_size>0 do
>   begin
>    BlockRead(F,file_buffer,Length(file_buffer),amt);
>    for l:=0 to amt-1 do
>     data[l+ptr]:=file_buffer[l];
>    file_size:=file_size-amt;
>    ptr:=ptr+amt;
>   end;
>   CloseFile(F);
>   AssignFile(F,filename+'test.dat');
>   ReWrite(F,1);
>   BlockWrite(F,data,fsize,amt);
>   CloseFile(F);
> end;


two bugs with this line.

1. Should be data[0].  You want to pass the start of the array, not the address 
of the variable that points to the start of the array.
2. fsize is <= 0 at this point.  You decrement it in your while loop

Changing it to

    BlockWrite(F, data[0], Length(data), amt);

worked for me.

-- 
Jeff Overcash (TeamB)
       (Please do not email me directly unless  asked. Thank You)
And so I patrol in the valley of the shadow of the tricolor
I must fear evil. For I am but mortal and mortals can only die.
Asking questions, pleading answers from the nameless
faceless watchers that stalk the carpeted  corridors of Whitehall.
              (Fish)

> 
> Doesn't explain why the data is corrupt later on - I added this write to 
> file to see what exactly was in the array - it has nothing to do with 
> what I'm trying to do.

Well Gerald,
It certainly explains why the ...test.dat file is different from the original.

> I have since changed the read file to:
> 
> AssignFile(F,filename);
> Reset(F,1);
> fsize:=FileSize(F);
> SetLength(data,fsize);
> BlockRead(F,data[0],fsize,amt);
> CloseFile(F);
> {Code follows to analyse the data}
> 
> As suggested by Jeff. The information in the array 'data' is not the 
> same as in the file loaded.

How did you see the difference? And what exactly is different? It is not enough to say 'it is not
the same', you need to be specific when describing problems. The code suggested by Jeff seems ok to
me.

> If I change the BlockWrite to data[0], I'm 
> sure it will work, but this won't fix the problem.

We first need to know exactly what the problem is. Then I'm sure we can find a cure.

Regards

-- 
Tom Brunberg
firstname.lastname@welho.com

On 19/01/2015 20:01, Tom Brunberg wrote:
> Gerald Holdsworth wrote:
>
>> On 19/01/2015 15:02, Tom Brunberg wrote:
>>> Gerald Holdsworth wrote:
>>>
>>
>>> Please show the procedure(s) where you load original file, create a new one (file or array?) and
>>> save it to disk.
>>
>> This has been concatenated for clarity. 'data' and 'fsize' are declared
>> in the public section of the Form declaration, while 'F' and 'amt' are
>> declared locally in the procedure. Also, the code is on the other
>> computer, which is currently turned off. If I change it to how Jeff
>> suggested (which is how the code now currently looks) it still works the
>> same.
>> The code to save the data is only temporary, and if I access the array
>> 'data' later on (which I do), I still get the corrupted data.
>>
>> var
>>    data: array of Byte;
>>    F: File;
>>    fsize, amt,ptr,file_size,l: Integer;
>>    file_buffer: array[0..$FFF] of Byte;
>> begin
>>    AssignFile(F,filename);
>>    Reset(F,1);
>>    fsize:=FileSize(F);
>>    file_size:=fsize;
>>    SetLength(data,fsize);
>>    ptr:=0;
>>    while file_size>0 do
>>    begin
>>     BlockRead(F,file_buffer,Length(file_buffer),amt);
>>     for l:=0 to amt-1 do
>>      data[l+ptr]:=file_buffer[l];
>>     file_size:=file_size-amt;
>>     ptr:=ptr+amt;
>>    end;
>>    CloseFile(F);
>>    AssignFile(F,filename+'test.dat');
>>    ReWrite(F,1);
>>    BlockWrite(F,data,fsize,amt);
>>    CloseFile(F);
>> end;
>
> You have an error, which is very easy to do.
> Change 'data' in this line to 'data[0]', like this:
>>    BlockWrite(F,data[0],fsize,amt);
>
> With dynamic arrays you need to address the first item, not the array.
> A dynamic array variable is a reference (pointer) to the actual array which is elsewhere in memory.
> A static array variable is the array itself.
>
> Cheers
>
Doesn't explain why the data is corrupt later on - I added this write to 
file to see what exactly was in the array - it has nothing to do with 
what I'm trying to do.

I have since changed the read file to:

AssignFile(F,filename);
Reset(F,1);
fsize:=FileSize(F);
SetLength(data,fsize);
BlockRead(F,data[0],fsize,amt);
CloseFile(F);
{Code follows to analyse the data}

As suggested by Jeff. The information in the array 'data' is not the 
same as in the file loaded. If I change the BlockWrite to data[0], I'm 
sure it will work, but this won't fix the problem.

Cheers,

Gerald.