Separate Service For Processing Long Running Tasks?
Jan 12, 2011
I would like to makea windows service. whenever the user of my ASP.NET application has to do a time-consuming task, the IIS would give the task to the service which will return a token(a temporary name for the task) and in the background the service would do the task. At anytime, the user would see the status of his/her task which would be either pending in queue, processing, or completed. The service would do a fixed number of jobs in parallel, and would keep a queue for the next-incoming tasks. In addition there would be a WinForms application for system administrator that would allow adding special ADMIn tasks such as "Clean orphaned files" or "archive data of inactive users".
Can you point me to something that can jump start me on this as a whole concept - I know I can google for windows services and I am able to do it myself from scratch but time is of the Essence so maybe you know of something that is already there and i can use block to build out of.
I'm looking for ways to improve a web page that initiates a long-running (>2 minutes) server-side task. The current version of the page just clocks for the full duration of the task, which can be very frustrating to the user.
I already have a few ideas about how I could improve the user's experience, but they all would involve the use of AJAX to some extent. Because of previous experiences that I've had on this project, I know that not all users have JavaScript enabled or available.
Assuming that the server-side process has already been optimized as much as possible, what else could I do to improve the experience of all users as much as possible?
I need to invoke a long running task from an ASP.NET page, and allow the user to view the tasks progress as it executes.
In my current case I want to import data from a series of data files into a database, but this involves a fair amount of processing. I would like the user to see how far through the files the task is, and any problems encountered along the way.
Due to limited processing resources I would like to queue the requests for this service.
I have recently looked at Windows Workflow and wondered if it might offer a solution?
I am thinking of a solution that might look like:
ASP.NET AJAX page -> WCF Service -> MSMQ -> Workflow Service *or* Windows Service
I have a long-running WCF service that I need to call, but I would like to do is to open modal window (modal popup extender) that simply shows progress and stops the user from interacting on the page until the service returns. What I was trying to do was the following:
1. Click button to activate the process which calls a method in my code-behind. 2. This method opens my modal panel with some pretty animation. 3. I call my WCF service asychronously so that the UI will refresh. 4. Service ends which calls my delegate I setup. 5. My delegate method would then refresh the page with results, and dismiss the model popup.
I know that similar questions have been asked all over the place, but I'm having trouble finding one that relates directly to what I'm after.
I have a website where a user uploads a data file, then that file is transformed and imported into SQL. The file could be up to 50mb in size, and some times this process can take 30 minutes or sometimes even longer.
I realise I need to palm off the actual work to another process, and poll that process on the web page. I'm wondering what the best approach would be though? Being a web developer by trade, I'm finding all this new Windows Service stuff a bit confusing, and I just wanted somewhere to start.
So:
Can I do / should I being doing this with a windows service? if so, how?
Should I use WCF? If this runs under IIS, will I have problems with aspnet_wp.exe recycling and timing out my process?
clarifications
The data is imported into sql, there's no file distribution taking place.
If there is a failure, it absolutely MUST be reported to the user. The web page will poll every, lets say, 5 seconds, from the time the async task begins, to get the 'status' of the import. Once it's finished another response will tell the page to stop polling for status updates.
queries on final decision
ok, so as I thought, it seems that a windows service is the best idea. So as to HOW to get it to work, it seems the 'put the file there and wait for the service to pick it up' idea is the generally accepted way, is there a way I can start a process run by the service, without it having to constantly be checking a database table / folder? As I said earlier, I don't have any experience with Windows Services - I wondered if I put a public method in the service, can I call it somehow?
Wondering if there is a performance difference between letting a long running process hang in asp.net vs running the process via a windows service. I have done this once before and the windows service was much quicker and didnt bog down my system, whereas the asp.net request seemed to wreak havoc.
I've developed a web application to accept video file uploads and then pass them to a backend service on an external server. The application runs without error on the visual studio debugging webserver, but once on a production iis 6 or 7 server, yields a timeout error at about a consistent amount of time into handling a large upload. Specifically, it errors in the middle of transferring the video file to the external server, once the application has successfully received it from the client. I'm aware of several timeouts to be configured related to the problem, and have done so. The application's web config has been tested with one or both of the following settings
<system.web> <httpRuntime executionTimeout="9999999" maxRequestLength="2048000" /> </system.web> and <configuration> <location path="default.aspx"> (the page at issue that's timing out) <system.web> <httpRuntime executionTimeout="9999999" maxRequestLength="2048000" /> </system.web> </location> </configuration>
And within the initialization of the webrequest made to the external server to send the video received from the client browser:
So with the execution time limits on both the webform as a whole and the connection made to the external server, I'm at a loss for what timeout is left unconfigured, or how to determine such, when I continue to get the following error: Unexpected error executing Brightcove Upload:...........................
in my case, It is the windows service which in running the long running insert/update on a table and I need to cancel the operation from my ASP.NET application. In the above link Burnsys suggests that one should kill the Sql server session. Is that really only way and a goood practice to do this? Also, in the same poset can use SqlCommand.Cancel to cancel. However, I am not sure how can I cancel the command from the windows service from ASP.NET application.
I am executing a long-running Oracle stored procedure from .NET. The procedure takes about three hours to run. Ideally, the user should be able to kick off the procedure, close the browser, and come back later to check the results.
The problem is that the connection to the Oracle procedure is lost after exactly an hour. As you would expect, the Oracle procedre runs to completion if it is executed from SQL Plus. Strangely enough, it will also run to completion if I run in debug mode on my local machine (I start two threads, one of which executes the procedure. I set a breakpoint on the second thread).
Here is my connection string:
data source= (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=serverx)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TestSID)))
We have a timer process (a JQuery plugin) that redirects after X number of minutes to the login page, all via JavaScript. When the timer hits zero, I want to run a task (could be anything; however, in this specific scenario, it's a web service call).
The issue I'm having is the web service that runs with the finish-up processes is not being called. The web service call happens, the redirect happens, I don't see any errors (I have try/catch statements around the setTimeout call), but no WS call.
When my Application face a long-time process, i.e fetch a query (SELECT a, b, c FROM d)This query needs 10 seconds to be completed in the MSSQL Management Studio, but when the ASP.NET application try to fetch it, it refuse to return any response to any other requests made on that Server.
I am hosting my Application on VPS Server with good specifications, and I am giving this example the (SELECT a, b, c FROM d) just to tell you the issue, it can be any process, maybe processing a movie, or even fetching some data through external API that is experiencing some slow-down,or whatever.
On my current project I have to accept 2 Excel spreadsheets that will be uploaded by clients, process them and create a download package based on this information. This process includes extracting data from the sheets, updating our databases and building several PDF files for the download package. This takes between 15 seconds to 2 minutes to complete, depending on the complexity of the request. Naturally, I want to show some kind of processing indicator rather than just leaving the user hanging while the page loads.
Here's the problem: How to show this processing indicator.
I have to do a full postback to upload the files so this eliminates several nice AJAX indicator methods (sponsoring users rejected the AJAX toolkit async file upload, saying it was confusing to them). If I process on any of the page events during the postback, the page doesn't load until the lengthy process is completed so the browser/site looks 'hung'.
Basically, I need some ideas on how to display a 'building your download' graphic while the lengthy process is working that will also work with a full postback.
I have an asp.net page that will be doing some processing that may take a very long time to complete. I cannot just set the page timeout value since this is going to be in a hosted environment and the timeout values that I set in my web.config are overridden by the server host. What I'm doing is taking a file from a FileUpload control and doing some web requests to a 3rd party service for each line in the file, all of which may take a very long time to process a large file. I'm talking on the order of, say, 30 minutes, and there's just no way to optimize this any further to cut down on the processing time. Is it possible to even do such a lengthy page request in asp.net? Can someone give me a pointer in the right direction here to make this happen? Is my only hope to create an async page? It seems that doing an async page is the way to go if I have a potential for a lot of lengthy requests, but really this massive of a request is going to happen VERY rarely so this is not an issue of running out of thread pool since most of the time this particular request will be completed relatively quickly, but on occasion it may receive a very large file it will need to process and will take a very long time. So what is the best way to handle that case?I'd also like to update the client with the processing status as the processing is going on. I'm familiar with doing client ajax calls via jQuery to a page webmethod so if there is some clean way to update the client as this long processing
i have a web application which which proccesses some request on some data that client selects, when the client selects more than 20 objects and clicks on proceed the client recieves this error, because the server takes a long time to process, however if the records are less and hence a timely response is recieved, no such error comes?
i have increased the sessiontimeout as well as set the
I have just started to look at the new "System.Threading.Tasks" goodness in .Net 4.0, and would like to know if there is any build in support for limiting the number of concurrent tasks that run at once, or if this should be manually handled.
E.G: If I need to call a calculation method 100 times, is there a way to set up 100 Tasks, but have only 5 execute simultaneously? The answer may just be to create 5 tasks, call Task.WaitAny, and create a new Task as each previous one finishes. I just want to make sure I am not missing a trick if there is a better way to do this.
In an asp.net web form, I keep getting a connection reset error message. The page is doing a some long running processing (about 2-5 minutes).
I have no problem when the web request comes from the same machine as the web server. But when the request originates across the network, I get a connection reset error about 1:30 or 2 minutes into waiting for a response.
I have set the in web.config for this application and put the application it's own application pool.
What else can I try?
Edit
The purpose of this page is to accept input from the user, calculate something, and send the result back to them. The long running calculation isn't something I can offload until a later time.
When testing this locally (on my dev machine) i see the application is working alot slower.
Is there a better way to write this code? Should i use ThreadPool.QueueUserWorkItem or create a new thread using Thread t = new Thread(new ThreadStart(DoWork)); ? Will it be better to create a totally seperate application for the purpose of sending the newsletters. will that help if ill run this application on the same machine?
i've seen other posts here talking about ThreadPool vs Thread but its seem no one is sure which is better.
3) Finally adding the reference of service contract project in a Hosting project (Console Application) with complete configurations.
I am adding one more windows app project as a client.When i am trying to add the service reference using the address "http://localhost:8000/Tasks", VS 2008 displays the below error.
There was an error downloading 'http://localhost:8000/Tasks'.
Unable to connect to the remote server
No connection could be made because the target machine actively refused it 127.0.0.1:8000
Metadata contains a reference that cannot be resolved: 'http://localhost:8000/Tasks'.
Could not connect to http://localhost:8000/Tasks. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:8000.
Unable to connect to the remote server
No connection could be made because the target machine actively refused it 127.0.0.1:8000
If the service is defined in the current solution, try building the solution and adding the service reference again.
When an ASPX page needs to make a call to a potentially long-running operation (lengthy DB query, call to a remote webservice, etc.), I use RegisterAsyncTask, so the IIS worker thread is returned to the pool, rather than being tied up for the duration of the long-running operation. However ASMX webservices don't have a RegisterAsyncTask function. When an ASMX webservice needs to call a potentially long-running operation, how can I implement the same behavior as RegisterAsyncTask? Note: the ASMX webservice is implemented as a script-service: returning json to a direct jQuery/ajax call. Therefore, I cannot use the "BeginXXX" approach described by MSDN, since that implements the asynchronous behavior within the generated client-stub (which isn't used when calling the webservice directly via ajax). EDIT: Adding source code: implemented the BeginXXX/EndXXX approach listed in John's answer. The synchronous "Parrot" function works fine. But the asynchronous "SlowParrot" function gives an internal server error: "Unknown web method SlowParrot"
Note:- i don't want to use updateprogress etc. control of ajax
on button click, long task(e.g thread) runs in my webpage for about 4-5 minutes.I want to show status to user either by a processing image through javascript(image must be shown in a certain part of page other part of page will remain intact) or an exact status of process if possible. i have tried a lot but all in vein.
I have a report that takes a couple of minutes to generate.What I am trying to do is display a progress indicatior onscreen (progress bar or spinning circle) while this is running.I was thinking of using javascript to display the progress indicator but am not sure how to get started on this. am using ASP.NET 2008, C#.
