I have inherited some code which was built with a older version of C++ Builder (XE5?) .  It has background processing done using the TApplication.OnIdle Event.  

When a dialog box is open, the OnIdle handler gets called repeatedly (which is what I want).  If no dialog box is open, the handler is never called.  I need the handler to be called regardless of the state of the program.  Any suggestions?

Thanks,

Curtis

Remy,

With your help, I almost have it working.  If I call HandleMessage multiple times before starting the timer, then both OnMessage and OnIdle get triggered.  I thought that OnMessage would be triggered when an message went in the queue and did not realize that calling HandleMessage was necessary.  This seems odd to me (and a co-worker) but as long as I can make it work, odd is OK.  

I am not sure when and how many time I need to call HandleMessage because I still do not understand the exact relationship between HandleMessage, the OnMessage handler and WndProc.  I am looking into this but if you have a link, that would be great.  

Thanks again,

Curtis

Remy,

I understand that idle is only triggered when the message queue goes from not empty to empty.  I understand that messages may be intercepted.  

Here are things that I would assume:
1)	Messages should either be intercepted or go to the OnMessage callback.
2)	If a message is intercepted, it should be removed from the queue.  Is this correct?
3)	If a message is handled by the OnMessage handler, it should be removed from the queue.
4)	If 1, 2 and 3 are true, then if I post a message, 
    a.	It will go on the queue.
    b.	It should be removed from the queue by something.
    c.	The queue should go from not empty to empty.
    d.	The OnIdle should be triggered.
But I am not getting the OnIdle even though the message appears to be sent correctly.  What I am missing?

I have read the posts by Raymond Chen and I have tried using:
	PostMessage instead of PostThreadMessage
	Using WM_NULL to try to get the queue to go empty and trigger the OnIdle.

In the code I am trying to get work I am not using HandleMessage. I am expecting either OnIdle or OnMessage to be triggered. I was using it only to try to understand what was going on in the queue.  I am trying to debug the HandleMessage as you suggested but it does not seem to be working.  I am still looking into this.

In the main thread, when the user clicks start, I have the code 
{code}
		tResult = CreateTimerQueueTimer( &hTimer, TIMER_QUEUE,
				(WAITORTIMERCALLBACK)TimerHandler, &threadId , TIMER_LENGTH, 0, WT_EXECUTEINIOTHREAD);
{code}

Here is the timer handler
{code}
void CALLBACK TimerHandler(HWND hwnd, UINT uMsg, UINT timerId, DWORD dwTime)

{
	DBG_PRINTF("TimerHandler");
	#ifdef POST_MSG
		DWORD err;
		UINT msg;
		msg = WM_USER;
		err = PostThreadMessage(MainThreadID,msg, 5, 5);
		DBG_PRINTF("PostThreadMessage %d to %d, err: %d %d", msg, MainThreadID, err,GetLastError());

		msg = WM_NULL;
		err = PostThreadMessage(MainThreadID ,msg ,0,0);
		DBG_PRINTF("PostThreadMessage %d to %d, err: %d %d", msg, MainThreadID, err,GetLastError());

		msg = WM_USER;
		err = PostMessage(Application->MainFormHandle,msg, 5, 5);
		DBG_PRINTF("PostMessage %d, err: %d %d", msg, err,GetLastError());

		msg = WM_NULL;
		err = PostMessage(Application->MainFormHandle,msg, 5, 5);
		DBG_PRINTF("PostMessage %d, err: %d %d", msg, err,GetLastError());

		err = SendMessage(Application->MainFormHandle,msg, 5, 5);
		DBG_PRINTF("SendMessage %d, err: %d %d", msg, err,GetLastError());

	#else
		bool Done;
		dummyThis->windowsIdle(NULL, Done);
	#endif

}
{code}

PostThreadMessage and PostMessage return an error value of 1.  SendMessage returns an error value of zero.

Message handler:
{code}
DLLEXPORT void __fastcall SYSTEM::OnMessageHandler(tagMSG &Msg, bool &Handled)
{

	DBG_PRINTF("+++++++++++++++OnMessage %d (0x%x)", Msg.message, Msg.message);
}
{code}

Idle Handler
{code}
DLLEXPORT void __fastcall SYSTEM::windowsIdle(TObject *Sender, bool &Done)
{
	DBG_PRINTF("--windowsIdle - Done %d", Done);
idleHook->activate(1, &Done);
}
{code}

Thank you,

Curtis

Curtis wrote:

> The appropriate solution to seems to be that I use CreateTimerQueueTimer()
> to trigger a callback in another thread.  The timer thread can then send a
> message via PostThreadMessage() to the main thread which can do the
> processing and update the display.

Yes.

> The problem I am having is that the line:
> 
> {code}
> err = PostThreadMessage(MainThreadID ,WM_USER+10 ,0,0);
> {code}
> 
> returns success (1) but the OnMessage callback does not get called..

As I explained earlier, there is only one way that can happen - something 
OTHER THAN TApplication is receiving the message.

An alternative approach would be to use PostMessage() instead of PostThreadMessage() 
(like Raymond Chen suggested in his blog).  You can use the Application->Handle 
window to post messages to, and then use the Application->OnMessage or TApplicationEvents::OnMessage 
event to receive them.  Or create a dedicated window using AllocateHWnd() 
instead.

> If something else was intercepting the message, I would expect Onldle
> to be called.

Only if Application->HandleMessage() sees an empty message queue.

> I know both of these callbacks are working because they are triggered
> when I open a dialog box.

That means absolutely nothing to me without seeing what you are seeing and 
doing.

> It seems to me that PostThreadMessage() is sending elsewhere or not
> really succeeding.

Not true on both counts.

> This might be a clue.  In trying to diagnose the problem, instead of
> the timer, I added the line
> 
> {code}
> Application->HandleMessage();
> {code}
> 
> to the main thread.

But WHERE did you add it?  What is triggering the call?

> The caused OnMessage to fire with a 2304 (0x900) message which
> makes sense.  But the OnIdle does not get triggered which does not.

Put a breakpoint on the HandleMessage() call and step into it with the debugger 
(turn on Debug DCUs in the project options).  Keep stepping and you will 
see why OnIdle is not being triggered.  You are likely to find that the message 
queue always has a message waiting when you are calling HandleMessage(). 
 That is the only way OnIdle would not be triggered.  So something in the 
main thread is pumping new messages into the queue, preventing the app from 
going idle.

> What else could be managing the message loop?

Modal dialogs, menu navigation, etc.  Things that require processing new 
UI messages while blocking your code.

> How would this been done?  How would I know if it occurring?

I cannot answe that without seeing some real code showing what you are actually 
doing.  It is very difficult to diagnose your problem when we canot even 
see what you are doing.

-- 
Remy Lebeau (TeamB)

Remy,

I appreciate you help and I feel I am making progress but it is not quite working.

The appropriate solution to seems to be that I use CreateTimerQueueTimer() to trigger a callback in another thread.  The timer thread can then send a message via PostThreadMessage() to the main thread which can do the processing and update the display.

The problem I am having is that the line:
{code}
err = PostThreadMessage(MainThreadID ,WM_USER+10 ,0,0);
{code}
returns success (1) but the OnMessage callback does not get called..  If something else was intercepting the message, I would expect Onldle to be called.  I know both of these callbacks are working because they are triggered when I open a dialog box.  

It seems to me that PostThreadMessage() is sending elsewhere or not really succeeding.  Do have any suggestions on how I could figure out what is failing?

This might be a clue.  In trying to diagnose the problem, instead of the timer, I added the line 
{code}
Application->HandleMessage();
{code}
to the main thread.  The caused OnMessage to fire with a 2304 (0x900) message which makes sense.  But the OnIdle does not get triggered which does not.

>>  only works if TApplication is managing the message loop. ….
>>...
>>The OnMessage event being triggered does not guarantee the message queue 
>>is empty. OnMessage is triggered whenever TApplication pulls a message off 
>>the top of the queue. Other messages can still exist in the queue, or be 
>>put into the queue while the pulled message is being processed.

What else could be managing the message loop?  How would this been done?  How would I know if it occurring?

Curtis

Curtis wrote:

> If something is swallowing messages, would I still get an OnIdle
> event?

Yes, if flow returns to the TApplication message loop and it then detects 
an empty message queue.  But if something else is running its own message 
loop without using TApplication.HandleMessage() to process messages than 
all bets are off.

> I added calls to Application->HandleMessage().

Where?

> This triggered the OnMessage handler which kind of makes sense
> but the message numbers did not 2304, 799, 49347 and lots of 512.

2304 is 0x0900, which is WM_USER + 1280, aka a message used by a private 
window class.  System-defined window class messages do not tend to go that 
high, so it is likely a private message used by a 3rd party UI control/library.

799 is 0x031F, which is a reserved system message.

49347 is 0xC0C3, which is message registered via RegisterWindowMessage() 
at runtime.

512 is 0x0200, which is a reserved system message.

> But the OnIdle did not trigger.

Then the message queue likely did not empty out.

> I guess I am still confused by the OnIdle handler.

I explained to you how it works.  But it only works if TApplication is managing 
the message loop.

> If OnMessage gets triggered, then I would expect OnIdle to be triggered
> immediately afterwards because the queue becomes empty.

The OnMessage event being triggered does not guarantee the message queue 
is empty.  OnMessage is triggered whenever TApplication pulls a message off 
the top of the queue.  Other messages can still exist in the queue, or be 
put into the queue while the pulled message is being processed.

-- 
Remy Lebeau (TeamB)

> The only way that should be happening is if the main thread message loop 
> is not receiving the message, such as if there is a modal OS dialog running 
> its own message loop that is swallowing messages.  Raymond Chen warned about 
> that several times in his "Old New Thing" blog on MSDN:
If something is swallowing messages, would I still get an OnIdle event?

I added calls to Application->HandleMessage().  This triggered the OnMessage handler which kind of makes sense but the message numbers did not 2304, 799, 49347 and lots of 512. But the OnIdle did not trigger.  

I guess I am still confused by the OnIdle handler.  If OnMessage gets triggered, then I would expect OnIdle to be triggered immediately afterwards because the queue becomes empty. 

Curtis

Curtis wrote:

> In my timer callback, I tried to post a message to the main thread
> using:
> {code}
> err = PostThreadMessage(threadId,WM_USER, 5, 5);

That should be fine, provided threadid is the same value as the global MainThreadID 
variable (which you could just use directly).  The TApplication message loop 
supports thread messages.

> err = PostMessage(Application->MainFormHandle,WM_USER, 5, 5);

You have to be careful with that.  The MainFormHandle property may end up 
reading the MainForm->Handle property, but the TWinControl::Handle property 
is not thread-safe.

> I expected to receive this in my OnMessage handler

Under normal conditions, yes you should.

> but it was not triggered.

The only way that should be happening is if the main thread message loop 
is not receiving the message, such as if there is a modal OS dialog running 
its own message loop that is swallowing messages.  Raymond Chen warned about 
that several times in his "Old New Thing" blog on MSDN:

Thread messages are eaten by modal loops
http://blogs.msdn.com/b/oldnewthing/archive/2005/04/26/412116.aspx

Watching thread messages disappear
http://blogs.msdn.com/b/oldnewthing/archive/2005/04/27/412565.aspx

Rescuing thread messages from modal loops via message filters
http://blogs.msdn.com/b/oldnewthing/archive/2005/04/28/412574.aspx

Why do messages posted by PostThreadMessage disappear?
http://blogs.msdn.com/b/oldnewthing/archive/2009/09/30/9901065.aspx

> I know the handler is functional because it is hit when I have a dialog box
> open.  BTW, you were correct OnIdle not firing because no messages are
> received empty during normal operation (i.e. the queue is always empty).

That goes back to suggesting the message(s) are being swallowed before the 
VCL sees them.

-- 
Remy Lebeau (TeamB)

I tried something else that seems like it should work but did not work either.

In my timer callback, I tried to post a message to the main thread using:
{code}
	err = PostThreadMessage(threadId,WM_USER, 5, 5);
	err = PostMessage(Application->MainFormHandle,WM_USER, 5, 5);
{code}
The err code is 1 indicating success.  

I expected to receive this in my OnMessage handler but it was not triggered.  I know the handler is functional because it is hit when I have a dialog box open.  BTW, you were correct OnIdle not firing because no messages are received empty during normal operation (i.e. the queue is always empty).

Thanks,

Curtis

Curtis wrote:

> You are correct that the timer callback is in a different thread.
> Do you know how to solve this problem?

Without seeing your actual code, the only thing I can suggest is make sure 
your timer callback code is thread-safe, and any direct UI access must be 
delegated to the main UI thread.  Any drawing that the timer wants to do 
can be done to an in-memory bitmap which the main thread can then draw onscreen 
when needed.

-- 
Remy Lebeau (TeamB)