Einzelnen Beitrag anzeigen

Kas Ob.

Registriert seit: 3. Sep 2023
347 Beiträge
 
#14

AW: Suche nach String mit 'decomposed' Character

  Alt 7. Okt 2024, 10:16
Hi,

In diesem Thread (Artikel #20) hatte ich bereits das Problem mit unterschiedlich codierten Umlauten thematisiert. Zur Zeit brüte ich über einer Suchfunktion, die in einem TMemo bei einer Suche nach 'Götterdämmerung' sowohl 'Götterdämmerung' (precomposed = Götterdämmerung) als auch 'Götterdämmerung' (decomposed = Götterdämmerung) findet.
Das ist wohl alles andere als trivial,
To be honest it is trivial ! but with the right tools.

Which is completely different APIs like FindNLSString and FindNLSStringEx
https://learn.microsoft.com/en-us/wi...-findnlsstring
https://learn.microsoft.com/en-us/wi...indnlsstringex
in fact all the functions the following link capable to perform what you want, with more or less tweaking parameters
https://learn.microsoft.com/en-us/wi...port-functions

Windows has National Language Support (NLS) and International Components for Unicode (ICU), but there is differences, ICU introduced to Windows 10 Creator, while NLS was there since Windows Vista (at least i think)

Anyway here is a simple test for you search problem and its solution, i will paste the code and attach the project because i can't trust the browsers to keep encoding right.

Delphi-Quellcode:
unit Unit11;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants,
  System.Classes, Vcl.Graphics, Vcl.Controls, Vcl.Forms, Vcl.Dialogs,
  Vcl.StdCtrls;

type
  TForm11 = class(TForm)
    Button1: TButton;
    Memo1: TMemo;
    Memo2: TMemo;
    procedure FormCreate(Sender: TObject);
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    procedure SearchStringList(AList: TStringList);
    procedure SearchStringFindNLSString(AList: TStringList);
    procedure SearchStringFindNLSStringEx(const LOCALE_NAME: string; AList: TStringList);
  end;

var
  Form11: TForm11;

implementation

{$R *.dfm}

const
  SUB_STR_1 = 'Götterdämmerung';
  SUB_STR_2 = 'Götterdämmerung';

procedure TForm11.FormCreate(Sender: TObject);
begin
  Button1.Click;
end;

procedure TForm11.SearchStringFindNLSString(AList: TStringList);

  procedure DoFindWithFindNLSString(const SubStr: string);
  var
    i, Res, Found: Integer;
  begin
    for i := 0 to AList.Count - 1 do
    begin // LOCALE_USER_DEFAULT = $400
      Res := FindNLSString(LOCALE_USER_DEFAULT, FIND_FROMSTART, PChar(AList.Strings[i]), -1, PChar(SubStr), -1, @Found);
      if (Res <> -1) and (Found > 0) then
        Memo1.Lines.Add(IntToStr(i));
    end;
  end;

begin
  Memo1.Lines.Add('finding lines with SUB_STR_1 = ' + SUB_STR_1);
  DoFindWithFindNLSString(SUB_STR_1);

  Memo1.Lines.Add('finding lines with SUB_STR_2 = ' + SUB_STR_2);
  DoFindWithFindNLSString(SUB_STR_2);
end;

procedure TForm11.SearchStringFindNLSStringEx(const LOCALE_NAME: string; AList: TStringList);

  procedure DoFindWithFindNLSStringEx(const SubStr: string);
  var
    i, Res, Found: Integer;
  begin
    for i := 0 to AList.Count - 1 do
    begin
      Res := FindNLSStringEx(PChar(LOCALE_NAME), FIND_FROMSTART, PChar(AList.Strings[i]), -1, PChar(SubStr), -1, @Found, nil, nil, 0);
      if (Res <> -1) and (Found > 0) then
        Memo1.Lines.Add(IntToStr(i));
    end;
  end;

var
  i, Res, Found: Integer;
begin
  Memo1.Lines.Add('finding lines with SUB_STR_1 = ' + SUB_STR_1);
  DoFindWithFindNLSStringEx(SUB_STR_1);

  Memo1.Lines.Add('finding lines with SUB_STR_2 = ' + SUB_STR_2);
  DoFindWithFindNLSStringEx(SUB_STR_2);
end;

procedure TForm11.SearchStringList(AList: TStringList);

  procedure DoFindWithPos(const SubStr: string);
  var
    i: Integer;
  begin
    for i := 0 to AList.Count - 1 do
      if Pos(SubStr, AList.Strings[i]) > 0 then
        Memo1.Lines.Add(IntToStr(i));
  end;

begin
  Memo1.Lines.Add('finding lines with SUB_STR_1 = ' + SUB_STR_1);
  DoFindWithPos(SUB_STR_1);

  Memo1.Lines.Add('finding lines with SUB_STR_2 = ' + SUB_STR_2);
  DoFindWithPos(SUB_STR_2);
end;

procedure TForm11.Button1Click(Sender: TObject);
var
  sList: TStringList;
  i: Integer;
begin
  sList := TStringList.Create;
  try
    sList.LoadFromFile('MacOS_ItunesContent_Small.txt');
    if sList.Count = 0 then
      Exit;
    for i := 0 to sList.Count - 1 do
      Memo2.Lines.Add(IntToStr(i) + #9 + sList.Strings[i]);

    Memo1.Lines.Add('Searching using Pos');
    SearchStringList(sList);
    Memo1.Lines.Add(#13#10'Searching with FindNLSString');
    SearchStringFindNLSString(sList);
    Memo1.Lines.Add(#13#10'Searching with FindNLSStringEx and LOCALNAME=''''');
    SearchStringFindNLSStringEx('', sList);

  finally
    sList.Free;
  end;
end;

end.
The data file in question, which is minimized and edited a little is MacOS_ItunesContent_Small.txt
Zitat:
dummy line

Götterdämmerung
Götterdämmerung
Track 4344
Track ID: 11572
Titel: Morgendämmerung und Siegfried's Rheinfahrt aus "Götterdämmerung"
Artist: Richard Wagner
Album: Rheingold
Track-Art: Abgeglichene AAC-Audiodatei
GUID:
Hinzugefügt: 14.12.2017 17:57:46
Persistent ID: 91453A3B4084EC74
Tracktype: File
Speicherort: Macintosh HD/Users/AlfonsYondraschek/Music/iTunes 1/iTunes Media/Music/Richard Wagner/Rheingold/05 Morgendämmerung und Siegfried's Rheinfahrt aus _Götterdämmerung_.m4a

Track 4345
Track ID: 11574
Titel: Siegfried's Trauermarsch und Finale aus "Götterdämmerung"
Artist: Richard Wagner
Album: Rheingold
Track-Art: Abgeglichene AAC-Audiodatei
GUID:
Hinzugefügt: 14.12.2017 17:57:46
Persistent ID: 6C45FF4271B8A57B
Tracktype: File
Speicherort: Macintosh HD/Users/AlfonsYondraschek/Music/iTunes 1/iTunes Media/Music/Richard Wagner/Rheingold/06 Siegfried's Trauermarsch und Finale aus _Götterdämmerung_.m4a
The result should be like this
Zitat:
Searching using Pos
finding lines with SUB_STR_1 = Götterdämmerung
2
6
18
finding lines with SUB_STR_2 = Götterdämmerung
3
14
26

Searching with FindNLSString
finding lines with SUB_STR_1 = Götterdämmerung
2
3
6
14
18
26
finding lines with SUB_STR_2 = Götterdämmerung
2
3
6
14
18
26
And here screenshot of the result for any cases
2024-10-07-11_02_02-untitled.png

The project with the data file
FindNLSString.zip

Notes on FindNLSString and FindNLSStringEx:
1) Although the documentation of FindNLSString advice to move to FindNLSStringEx, yet Notpad.exe is using FindNLSString !
2) FindNLSStringEx use LOCALE_NAME (plain string) instead of the structured LOCALE_NAME value, yet it is go complicated very fast when you need to chain many languages, so the sticking to default (USER or SYSTEM) is easier, in that case it is better and easier to use FindNLSString.
3) implementing similar algorithm in pure Pascal is huge job unless you will depend on either NLS or the ICU library, well .. such dependency will render any implementation useless and waste of time.

Hope that help
Kas
  Mit Zitat antworten Zitat