Code Optimization

[sdean]

Hi i've these functions to Store Streams in One File :

Delphi-Quellcode:

function AddData(ID:string; pBuf: Pointer; Count: Integer): Integer;
var
  Buff   : Pointer;
begin
 // FFile is of Type : TFileStream
  FFile.Seek(0, soFromEnd);//--> the Container * 
  if( Result > -1 )then
  begin
   GetMem(Buff, Count);
   try
    Move(pBuf^, Buff^, Count);
    Result := FFile.Write(Buff^, Count);
   finally
    FreeMem(Buff, Count);
   end;
  end;
end;
{* and here how i add Streams *}
function InsertStream(ID: String;aStream: TStream): Integer;
var
  pBuff : Pointer;
begin
  // alloc Mem for pBuff
  GetMem(pBuff, aStream.Size);
  try
   aStream.Position := 0;
   aStream.Read(pBuff^, aStream.Size);
   // now add data
   Result := AddData(ID, pBuff, aStream.Size);
  finally
   FreeMem(pBuff, aStream.Size);
  end;
end;

my question is :
how could i optimize the InsertStream Function so that it can work fast for file's size more than 1 GB cause i cannot load such a huge size in Memory.


many thanks in advance

[FredlFesl]

The Code contains errors and redundant stuff.
Here's my improved / modified and untested version.

Delphi-Quellcode:

Const
  BLOCKSIZE = 1024*1024; // Write in chunks @ 1MB

function AddData(ID:string; pBuf: Pointer; Count: Integer): Integer;
begin
  FFile.Seek(0, soFromEnd);
  if( Result > -1 )then // ***** WHERE DOES THIS Result COME FROM? if you leave it Like this, it will not work.
    Result := FFile.Write(PAnsiChar(pBuf)^, Count);
end;

function InsertStream(ID: String;aStream: TStream): Integer;
var
  pBuff : Pointer;
  BytesRead : Integer;

begin
  GetMem(pBuff, BLOCKSIZE); // allocate a chunk of memory
  try
   aStream.Position := 0;
   Repeat
     BytesRead := aStream.Read(pBuff^, MIN (BLOCKSIZE, aStream.Size - aPosition)); // move data into the chunk
     Result := AddData(ID, pBuff, BytesRead); // Add the chunk to the file
   Until (Result<>0) or (BytesRead < BLOCKSIZE);
  finally
   FreeMem(pBuff, BLOCKSIZE);
  end;
end;

However, a much simpler approach would be

Delphi-Quellcode:

FFile.Seek(0, soFromEnd);
FFile.CopyFrom (aStream, FFile.Position);

[sdean]

thank you FredlFesl

Delphi-Quellcode:

BytesRead := aStream.Read(pBuff^, MIN (BLOCKSIZE, aStream.Size - aPosition));

aPosition >> do you mean aStream.Position

[Luckie]

Why is FFile global? If it is global because it is needed in different parts of the code, why don't you make a class out of the code?

[sdean]

thank you Luckie :
any help of that !

[FredlFesl]

Zitat:

Zitat von sdean (Beitrag 1099856)

thank you FredlFesl

Delphi-Quellcode:

BytesRead := aStream.Read(pBuff^, MIN (BLOCKSIZE, aStream.Size - aPosition));

aPosition >> do you mean aStream.Position

Yes.

[Luckie]

The class could look like this:

Delphi-Quellcode:

TXyz = class
private
  FFile: TFileStream;
public
  property XyzFile: TFileStream read FFile write FFile;
  procedure AddData(...);
  procedure InsertData(...);
end;

[sdean]

but FredlFesl : your improved / modified and untested version writes Only 1MB to FFile
i mean the FFile will only hold 1MB of the Source Stream not the Whole Stream size .

[sdean]

Zitat:

Zitat von Luckie (Beitrag 1099864)

The class could look like this:

Delphi-Quellcode:

TXyz = class
private
  FFile: TFileStream;
public
  property XyzFile: TFileStream read FFile write FFile;
  procedure AddData(...);
  procedure InsertData(...);
end;

Thank you Luckie , Could you please explain :
what s the Diff between Declaring FFile as global and Creating this New Class as long as the Result will remain the same : the Write / Read Operations will be focusing on the FFile .

and many thanks .

[Luckie]

The result will of course be the same. But encapsulating the code makes it more readable, easier to administrate and (re)use.