Delay In Silverlight Initialization In Aspx Page?

We have a silverlight/asp.net application which communicates with WCF to fetch data. Now we are facing a problem where in the silverlight component is taking some time to initialize after the asp.net page life cycle is completed.We have tried tracing all the events and found that there is a time lapse between the aspx page unload event and silverlight initialize event. This we have tried with even a simple application (hello world) but still have found the same result.There is nearly 3-4 seconds delay i.e the silverlight component initialization starts 3-4 seconds after the page unload event ends.