Hi,

we are in the process of migrating an old midas/com server project to the new DataSnap orientation.

Trying to not confuse things (as they are so different approaches), we've started a server project from scratch, and then move some of the business logic (all that apply) to the new server.

The main goal is to split the server logic into various server-modules, according to functionality... 

So: 

- all that has to do with banking accounts would lay in a module 
- anything involving sales would be in a separated module
- employees and payroll (guess) another module...
and so on...

those modules may share the same database or could involve separated databases (interbase by now).

Right now the application runs in local networks. But the next main goal is to escalate it, so it could work over internet too.

The structure the is now as following:

- Server Side 
  * set as a windows service : TService;
  * 1         : TDSServer;
  * 1         : TDSTCPServerTransport;
  * 1         : TDSHTTPService (thinking about internet)
  * several : TDSServerClass (one for each TDSServerModule - wich are server methods)
  * the server methods (TDSServerModules) have a lot of tIbQuerys, tDatasetProviders, and TIBStoredProc (using ibExpress)
  * the database(s) name(s) and connection info is loaded at run-time in the main TDSServerModule.OnCreate, from a settings file;

- Client Side
  * 1         : TSQLConnection
  * 1         : Proxy unit which I use to call server methos on the client side
  * several : TDSProviderConnection (one for each server method)
  * several : TDatamodule (to split the logic on client side too)
  *  the TDataModules has a lot of TClientDatasets, TDatasources an so on.

That's the way we've understand the white papers, samples and videos from Dr. Bob, Paweł Głowacki (to name some) and many many others we have read and see.

This way things kind of work, but we are facing many (worrying)  issues:

- after the client connect to the main TDSServerClass, we MUST always call a log procedure we published on the server, otherwise we get the "database not assigned" error; a clear sign that the OnCreate event on the main TDSServerModule did not fire on connection.


- we HAD to set all the TDSServerClass.LiveCycle from "session" to "server", because when in "session" 2 clients connected and one of them disconnected the other one client rises the exception "DATABASE NOT ASSIGNED" which is weird; cause as our understanding, when in "session" the clients should have independent instances from the server....

- For some reason, when we first compiled the server with XE6 the data posted with applyUpdates where (almost always) just in the server side memory and where not committed to the database (we had to compile the server side in D2010 to avoid that BIG-problem)

THE LIST COULD GO ON, but to summarize: WE GOTTA BE MISSING SOMETHING..

Maybe we are lost in the path, or are still thinking focused on the old midas/com approach.

That said, we would like some real-world multiple TDSServerClass / TDSServerModules example, some connection or approach suggestions, database components suggestions, some in-depth reading that could clarify our minds, anything. (not the a-b-c examples available online please).

We had relayed on the old Midas/Com server for years with just a few minor complaints, but now we feel like we are (and the data) floating in some kind of lost dimension without any clear direction....

Please, help!!!

Hi Moises,

With three different Interbase databases and a MySql database it is quite complex application!

You have all database connection components located on main DSServerModule. How do you link data access components located on other DSServerModules to the database connection? Is it set by design-time perhaps? There could be problem. Setting the database connection at runtime is preferable.

You could experiment with dropping on every DSServerModule their own database connection components. I think it will work, but you have more open connections to database.

Or create function in ServerContainer to create or reuse database connection component regarding if same session or not, like in previous mentioned article of Andreano Lanusse.
http://www.andreanolanusse.com/en/sharing-db-connection-between-multiples-datasnap-server-modules/

Merry XMas!

Best regards,
Erwin

Hi,

> {quote:title=Erwin Mouthaan wrote:}{quote}
> Hi Moises,
> 
> If LifeCycle is Session then indeed TDSServerClass instance last until client disconnects, and is isolated / independent for each client. The instance itself is isolated from each client, not global variables it references. Where on server-side is your database connection component instantiated? In OnCreate event of DSServerClass? Or is it global variable on a DataModule perhaps?
> 

In the main TDSServerModule we have 3 TIBDatabase components (for interbase databases) and 1 tSQLConnection component (for MySql Database) we set the parameters for those components in the OnCreate event of this same TDSServerModule.


> Obviously your problems have something to do with database connection component. Is your database connection component really a member of your TDSServerClass, instantiated in OnCreate and released in OnDestroy event? For DataSnap to be thread-safe every client has to use it's own database connection component.
> 

In the server container we have the tDSServerClass.OnGetClass event like this:
{code}
procedure TFinanciera_Main_ServerContainer.Main_Connections_ClassGetClass(
  DSServerClass: TDSServerClass; var PersistentClass: TPersistentClass);
begin
  PersistentClass := MainRemote_SM.TMain_ServerMethods; {this is the main DSServerClass, where the database connection components are}
end;
{code}
Then... in the TDSServerModule.OnCreate we have this:
{code}
procedure TMain_ServerMethods.DSServerModuleCreate(Sender: TObject);
Var
  I     : Integer;
begin
      _Log_Module_Name := 'Main Data Server';                 {_Log_Module_Name IS a global variable}

      If _ProperConfig Then                                                 {_ProperConfig IS a global variable set at service start if the settings file is OK}
        Begin
          SystemLog (_Log_File_Name,_Log_Module_Name,now,'Configuration OK');           {SystemLog is UDF to log server runtime information}
          SystemLog (_Log_File_Name,_Log_Module_Name,now,#32);
          SystemLog (_Log_File_Name,_Log_Module_Name,now,'Starting DB Configuration');
          tConfigDataset.LoadFromFile(_ConfigFileName);             {tConfigDataset is a TclientDataset that loads an .xml / _ConfigFileName is a global variable set at service start} 

          For  I := 0 To Self.ComponentCount - 1 Do
            Begin
              If Not (Self.Components[I] Is TIBDatabase) And
                 Not (Self.Components[I] Is TSQLConnection) Then Continue;

              If Not (tConfigDataset.Locate('Module',Components[I].Name,[])) Then {If not found the TIBDatabase in the .xml then no configuration is provided}
                Begin
                  SystemLog (_Log_File_Name,Components[I].Name,now,'No settings for this DB');
                  Continue;
                End;

              If Not (tConfigDatasetActive.Value) Then {there is a field in the .xml to say if DB is going to be used or not}
                Begin
                  SystemLog (_Log_File_Name,Components[I].Name,now,'DB no Active');
                  Continue;
                End;

              If (Self.Components[I] Is TIBDatabase) Then
                Begin
                  _Log_Module_Name := (Self.Components[I] As TIBDatabase).Name;
                  Try {Config}
                    SystemLog (_Log_File_Name,_Log_Module_Name,now,'Setting DB Connection');
                    (Self.Components[I] As TIbdatabase).LoginPrompt := False;

                    (Self.Components[I] As TIbdatabase).Params.Clear;

                    (Self.Components[I] As TIbdatabase).DatabaseName := UnEnCryptString(tConfigDatasetDb_Address.AsString,
                                                                                        _Encrypt_Password);
                    SystemLog (_Log_File_Name,Components[I].Name,now,(Self.Components[I] As TIbdatabase).DatabaseName);

                    (Self.Components[I] As TIbdatabase).Params.Add('user_name='+UnEnCryptString(tConfigDatasetUserName.AsString,
                                                                                                _Encrypt_Password));

                    (Self.Components[I] As TIbdatabase).Params.Add('password='+UnEnCryptString(tConfigDatasetPassword.AsString,
                                                                                               _Encrypt_Password));

                    (Self.Components[I] As TIbdatabase).Params.Add('sql_role_name='+tConfigDatasetRole.AsString);
                    SystemLog (_Log_File_Name,_Log_Module_Name,now,'DB Settings OK')
                  Except
                    On E:Exception Do SystemLog (_Log_File_Name,_Log_Module_Name,now,'SETTINGS ERROR : ' + E.Message)
                  End
                End;

              If (Self.Components[I] Is TSQLConnection) Then {for a MySql Database}
                Begin
                  _Log_Module_Name := (Self.Components[I] As TSQLConnection).Name;
                  Try {Config}
                    SystemLog (_Log_File_Name,_Log_Module_Name,now,'Setting DB Connection');
                    (Self.Components[I] As TSQLConnection).LoginPrompt := False;

                    (Self.Components[I] As TSQLConnection).Params.Clear;
                    (Self.Components[I] As TSQLConnection).ConnectionName := tConfigDatasetConnectionName.AsString;
                    (Self.Components[I] As TSQLConnection).DriverName := tConfigDatasetDriver.AsString;
                    (Self.Components[I] As TSQLConnection).GetDriverFunc := tConfigDatasetGetDriverFunc.AsString;
                    (Self.Components[I] As TSQLConnection).LibraryName := tConfigDatasetLibraryName.AsString;
                    (Self.Components[I] As TSQLConnection).LoginPrompt := False;
                    (Self.Components[I] As TSQLConnection).VendorLib := tConfigDatasetVendor.AsString;

                    (Self.Components[I] As TSQLConnection).Params.add('DriverName='+tConfigDatasetDriver.AsString);
                    (Self.Components[I] As TSQLConnection).Params.add('HostName='+tConfigDatasethostname.AsString);

                    (Self.Components[I] As TSQLConnection).Params.add('Database='+UnEnCryptString(tConfigDatasetDb_Address.AsString,
                                                                                                  _Encrypt_Password));

                    (Self.Components[I] As TSQLConnection).Params.add('User_Name='+UnEnCryptString(tConfigDatasetUserName.AsString,
                                                                                                   _Encrypt_Password));

                    (Self.Components[I] As TSQLConnection).Params.add('Password='+UnEnCryptString(tConfigDatasetPassword.AsString,
                                                                                                  _Encrypt_Password));

                    (Self.Components[I] As TSQLConnection).Params.add('BlobSize='+tConfigDatasetBlobSize.AsString);
                    (Self.Components[I] As TSQLConnection).Params.Add('Role='+tConfigDatasetRole.AsString);
                    (Self.Components[I] As TSQLConnection).Params.Add('Port='+tConfigDatasetport.AsString);
                    SystemLog (_Log_File_Name,_Log_Module_Name,now,'DB Settings OK')
                  Except
                    On E:Exception Do SystemLog (_Log_File_Name,_Log_Module_Name,now,'SETTINGS ERROR : ' + E.Message)
                  End
                End
            End
        End
end;
{code}

Thanks,
Moises

Hi Moises,

If LifeCycle is Session then indeed TDSServerClass instance last until client disconnects, and is isolated / independent for each client. The instance itself is isolated from each client, not global variables it references. Where on server-side is your database connection component instantiated? In OnCreate event of DSServerClass? Or is it global variable on a DataModule perhaps?

Obviously your problems have something to do with database connection component. Is your database connection component really a member of your TDSServerClass, instantiated in OnCreate and released in OnDestroy event? For DataSnap to be thread-safe every client has to use it's own database connection component.

Best regards,
Erwin

Hi,
Thanks for answering...

> {quote:title=Erwin Mouthaan wrote:}{quote}
> Hi,
> 
> You speak about client connecting to main TDSServerClass, but not exactly clear what you mean. Maybe problem there.
>

That was a miss-writing, sorry, it's really as you pointed... we connect the TSQLConnection by calling TSQLConnection.Open;

> 
> Anyway, when at client-side SQLConnection opens, a connection is only made to TDSServer, not with an instance of a TDSServerClass. Only > when at client-side a server method is invoked through proxy DataSnap classes, an instance of TDSSServerClass is created. 
>

It took me a while to understand that (as I got used to just open the TSocketConnection)...  that's why we call a logclient server method rigth after the TSQLConnection is connected.

>And immediately destroyed when TDSServerClass.LifeCycle property is set to Invocation and server method has finished. When set to Session >the instance of the DSServerClass is destroyed when the SQLConnection at client-side closes.
> 
> Other client connection has other session, and each has own instance of TDSServerClass when DSServerClass.LifeCycle property has value Session. Same DSServer, but different DSServerClass instance.
> 

Here is where things go weird..... 

1- we understand LifeCycle this way:

- Invocation -> TDSServerClass Instance last just as long as the client request something;
- Session ->TDSServerClass Instance last until client disconnect - and is isolated / independent  for each client;
- Server -> TDSServerClass Instance last while there are any client connected - and is shared for every client;

if so... when we set to Session ... why this happen?:

clients connect simultaneously in different PC's, If the first connected client disconnects, then the other clients (on the next DB request) raises the error "Database Not assigned"? (like the Instance was shared and destroyed when the first client got out) 

Keep in mind here that we load the Database Information from a settings file, and we load it in the OnCreate of the main TDSServerClass 

The only workaround we found to stop that error from raising was set LifeCycle to Server. 

2- and most worrying...

We first compiled the server-side using Delphi XE6 and it was all ok... until 1 day we realized that a transaction made on one client was NEVER visible to other clients (even after applyUpdates and refresh).

After a lot of checking we noted that ALL transactions were left at server's memory. That day after stopping the service, the transactions where posted to the database, but the next time we just lost a huge lot of transactions.

We checked the forced-writes of the Database but that did not help...

workaround: quickly remove the dust from the good old Delphi 2010, remove all things not supported and compile the server there....
so now we have server compiled on D2010 and client compiled on DXE6.. 

Those things include, session manager, and transport filters.... which took a lot of time to know it was the reason why raised the error "Connection Gracefully Refused" or something like that.

> If you want to share database connection at server-side between different TDSServerModule, take look at article by Andreano Lanusse:	 Sharing DB connection between multiples DataSnap Server Modules.
> http://www.andreanolanusse.com/en/sharing-db-connection-between-multiples-datasnap-server-modules
> 

I just read the link.... thanks... now I have to do the test using that functions....

I'll let you know..


Thanks,
Moises

Hi,

You speak about client connecting to main TDSServerClass, but not exactly clear what you mean. Maybe problem there.

Anyway, when at client-side SQLConnection opens, a connection is only made to TDSServer, not with an instance of a TDSServerClass. Only when at client-side a server method is invoked through proxy DataSnap classes, an instance of TDSSServerClass is created. And immediately destroyed when TDSServerClass.LifeCycle property is set to Invocation and server method has finished. When set to Session the instance of the DSServerClass is destroyed when the SQLConnection at client-side closes.

Other client connection has other session, and each has own instance of TDSServerClass when DSServerClass.LifeCycle property has value Session. Same DSServer, but different DSServerClass instance.

If you want to share database connection at server-side between different TDSServerModule, take look at article by Andreano Lanusse:	 Sharing DB connection between multiples DataSnap Server Modules.
http://www.andreanolanusse.com/en/sharing-db-connection-between-multiples-datasnap-server-modules

Best regards,
Erwin