$0 in stream and loading as text

[WojTec]

I have sample data in stream like shown on attached image. You see many $0 inside. I want to load this stream to TStrings:

Delphi-Quellcode:

Text.LoadFromStream(Data);

Yeah, loading, but only to first $0. How to load whole data of this stream as text?

I tried also use additional string stream with encoding as Ansi or Unicode on create and load to strings from this encoded stream. The same result.

[Dejan Vu]

Load as binary (TFileStream) and then cast as string. But I am afraid it won't make you happy.

[Sir Rufo]

Which first $0 are you talking about?

The stream contains data with a MultiByte-Character-Set. When talking about $0 is this the value of the char or the value of the byte (half part of a char)?

Please be more precise in your questions

[himitsu]

Zitat:

Zitat von Sir Rufo (Beitrag 1269701)

The stream contains data with a MultiByte-Character-Set.

Encoding: UCS-2 Little Endian or UTF-16 :stupid:

[WojTec]

Zitat:

Zitat von Sir Rufo (Beitrag 1269701)

Which first $0 are you talking about?

I'm talking about stream content (bytes). First 2 bytes are $64 (100d = Ascii(D)Ascii(d)) and $00. I'm trying load this stream as text (above code), in sample stream, after load to TStrings I have 'D''d', so looks they stop decoding text when see $0.

When I save data and load in notepad, I see characters separated with space. I want to decode whole stream as text, even if $0 bytes inside.

TStringStream.Create('', TEncoding.Unicode)
TStringStream.LoadFromStream(Source);
TStrings.LoadFromStream(TStringStream);
// result = samplestream('D''d')

[Sir Rufo]

Zitat:

Zitat von WojTec (Beitrag 1269705)

Zitat:

Zitat von Sir Rufo (Beitrag 1269701)

Which first $0 are you talking about?

I'm talking about stream content (bytes). First 2 bytes are $64 (100d = Ascii(D)) and $00. I'm trying load this stream as text (above code), in sample stream, after load to TStrings I have 'D', so looks they stop decoding text when see $0.

When I save data and load in notepad, I see characters separated with space. I want to decode whole stream as text, even if $0 bytes inside.

TStringStream.Create('', TEncoding.Unicode)
TStringStream.LoadFromStream(Source);
TStrings.LoadFromStream(TStringStream);
// result = samplestream('D')

Oh, you are in a bigger trouble than I thought. You should have seen d and not D.
So maybe yor compiler or compter is broken ... sorry to tell that

[WojTec]

Zitat:

Zitat von Sir Rufo (Beitrag 1269706)

Oh, you are in a bigger trouble than I thought. You should have seen d and not D.
So maybe yor compiler or compter is broken ... sorry to tell that

Probably I see d, not D :cyclops:
Anyway don't working annd changing encoding don't want to help :?

[Sir Rufo]

Zitat:

Zitat von WojTec (Beitrag 1269707)

Zitat:

Zitat von Sir Rufo (Beitrag 1269706)

Oh, you are in a bigger trouble than I thought. You should have seen d and not D.
So maybe yor compiler or compter is broken ... sorry to tell that

Probably I see d, not D :cyclops:
Anyway don't working annd changing encoding don't want to help :?

Maybe you get now the point about "be more precise" ;)
It is near to impossible to help if you tell us about something like.

The source did not contain a whole text but a bunch of strings in a kind of structure. You have to work on that structure and read the data following the structure rules.

A good starting point is to look at the producer of the data and get some information. We do not have (because you did not tell us - another precise mark) any clue about the producer.

[brechi]

You have to write your own component to visualize the text.
Internal PAnsiChar/PWideChar is used and a #0 is the end of the text.

So if you use

Code:

  Memo1.Lines.Text := 'test1'#0'test2'#0;

only the "test1" will be shown.

If you dont want to save your data, then you can load the file into a TMemoryStream (or a string) and replace the #0 with another character.

Code:

var
  s: String;
  fm: TFileStream;
fm := TFileStream.Create;
fm.LoadFromFile('sadasd', fmOpenRead);
setLength(s, fm,size);
fm.Read(s[1], Length(s));
s := StringReplacte(s, #0, #1, [rfReplaceAll]);
memo1.lines.text := s;
etc.

[jaenicke]

Zitat:

Zitat von brechi (Beitrag 1269781)

If you dont want to save your data, then you can load the file into a TMemoryStream (or a string) and replace the #0 with another character.

This is only unicode text, so the problem is not to replace those bytes, but to read the file with correct encoding.
If one did it this way and there is only one real unicode character inside, one would get really strange results...

Even if there is no byte order mark this code works perfectly for me:

Delphi-Quellcode:

var
  test: TStringStream;
begin
  test := TStringStream.Create('', TEncoding.Unicode);
  try
    test.LoadFromFile('c:\temp\a.txt');
    ShowMessage(test.DataString);
  finally
    test.Free;
  end;
end;

So it would be better, if you would attach a working sample code with an example file...

[WojTec]

@Sir Rufo, next time I'll be more precise :)
Data producer is unknown, I just know it is some resource compiler + compiler + linker (mean file is executable). Data is some data, it can be in any format, it can be a string too.

I wrote simple function to check if stream can be Unicode or ANSII:

Delphi-Quellcode:

function GetEncoding(AStream: TStream): TEncoding;
var
  V: Byte;
begin
  Result := TEncoding.ANSI;

  if AStream.Size > 1 then
  begin
    AStream.Position := 1;
    AStream.ReadBuffer(V, 1);
    if V = 0 then
      Result := TEncoding.Unicode
    ;
  end;
end;

Another one, I badly loaded encoded text (LoadFromStream - it was raw data, stupilo :oops:), Append(DataString) is required.

As @brechi told, pointer terminate code is zero in TStrings, so this is main problem. I don't know how jump over it :(

Also I wrote poor (because very slow) procedure to replace bytes in stream, if someone need it:

Delphi-Quellcode:

procedure ReplaceBytes(AIn: TStream; const AFrom, ATo: Byte; AOut: TStream);
var
  I: Cardinal;
  V: Byte;
begin
  for I := 1 to AIn.Size do
  begin
    AIn.ReadBuffer(V, 1);

    if V = AFrom then
      AOut.WriteBuffer(ATo, 1)
    else
      AOut.WriteBuffer(V, 1)
    ;
  end;
end;