C# - Throttle Parallel Request To Remote Api?
Jul 14, 2010
I'm working on an ASP.NET MVC application that uses the Google Maps Geocoding API. In a single batch there may be upto 1000 queries to submit to the Geocoding API, so I'm trying to use a parallel processing approach to imporove performance. The method responsible for starting a process for each core is:
public void GeoCode(Queue<Job> qJobs, bool bolKeepTrying, bool bolSpellCheck, Action<Job, bool, bool> aWorker)
{
// Get the number of processors, initialize the number of remaining
// threads, and set the starting point for the iteration.
int intCoreCount = Environment.ProcessorCount;
int intRemainingWorkItems = intCoreCount;
[Code]...
This is based on patterns document I found on Microsoft's parallel computing web site. The problem is that the Google API has a limit of 10 QPS (enterprise customer) - which I'm hitting - then I get HTTP 403 error's. Is this a way I can benefit from parallel processing but limit the requests I'm making? I've tried using Thread.Sleep but it doesn't solve the problem.
View 1 Replies
Similar Messages:
Apr 20, 2010
Currently we are developing an ASMX, ASP 2.0, IIS 7 web service that does some calculations (and return a dynamically generated document) and will take approx. 60 seconds to run.Since whe have a big machine with multiple cores and lots of RAM, I expected that IIS tries its best to route the requests that arrive in its requests queue to all available threads of the app pool's thread pool.But we experience quiet the opposite:When we issue requests to the ASMX web service URL from multiple different clients, the IIS seems to serially process these requests. I.e. request 1 arrives, is being processed, then request 2 is being processed, then request 3, etc
View 2 Replies
Sep 10, 2010
I want to perform a get request on a remote URL and then extract the headers returned.
View 1 Replies
Mar 28, 2011
I'm using a custom ashx handler to handle a file upload. When run locally, the file uploads fine.When I use the same setup on the web server I get a "Index out of range" error.In firebug I see the binary contents of the file in the post data and the file name is also passed in the query string.Any one seen this before?I`m sure its something minor, but its driving me up the wall.
Request header:
Key Value
Request POST /Secured/UploadHandler.ashx? HTTP/1.1
Accept text/html, application/xhtml+xml, */*
Referer http://cms.webstreet.co.il/Secured/fileUpload.aspx
Accept-Language he-IL
User-Agent Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
Content-Type multipart/form-data; boundary=---------------------------7db13b13d1b12
Accept-Encoding gzip, deflate
[code]...
View 2 Replies
Aug 9, 2010
I move a file from server to another server .in local it is woking fine .but in production it is getting error
The remote server returned an error: (408) Request Timeout.
i used like this ..
objWebClient.UploadFile(ClientLocation,
"POST", fileName.ToString());
View 3 Replies
Jul 26, 2010
There is one page which is actually a streaming to The Axis IP camera which spits MJPEG output.It requires user to log in with the user name/password promp on browser .I am using this stream to show video directly on a web page.It shows video correctly but asks user to provide correct user name and password set for the camera,I tried to logging in to this camera on server side using HTTP requests and then I realized I authenticated server request not the browser the end user is using.
So what I want is a method server side or client side, that can allow me to log-in to camera automatically when my end-users visit this page.I am using asp.net with c# 2005
View 1 Replies
Jan 11, 2011
I have a web service hosted on a web server, I invoke the web service using jquery ajax.
The service returns results successfully when invoked locally from the server, but it fails when invoked from a remote client machine (not in the same domain).
I see the request fail in the firebug returning error (401 UnAuthorized) and the response has the following error (Request format is unrecognized for URL unexpectedly ending in '/List').
View 3 Replies
Jun 19, 2010
I am calling a remote service and authenticating using a certificate. When testing with a Console App, everything works fine. When calling from an ASP.NET Website (.NET 4.0, IIS7) I receive a response code of 401 -- Unauthorized.I am adding the certificate using code such as:
var client = new TheGeneratedProxy();
client.ClientCertificates.Add(new X509Certificate("D:cert.pfx", "myPassword"));
(NOTE: I have also loaded the .pfx into the local Certificate Store using IE. The certificate is loaded into my "Personal" store -- so I suspect this to be the problem, since the Website will be running under a different account.)
View 1 Replies
Aug 7, 2010
using oAuth.net library for the Gmail Contacts API The request seems well-formed and I have stepped through the requests, and cannot discern anything wrong. Tried checking if the timestamp is an issue - it seems that was a common issue on the gmail forums, tried changing to various time-zones, still no luck.
public string WebResponseGet(HttpWebRequest webRequest)
{
StreamReader responseReader = null;
string responseData = "";
[Code]....
View 1 Replies
Aug 3, 2010
When i try to execute the code below i get the error "unable to connect to remote server".
The internet connection is set and if i copy and paste the serviceurl directly on the browser it works.
public string GeoCode(string address)
{
if (String.IsNullOrEmpty(address)) throw new ArgumentNullException("address");
string ServiceUrl = "http://maps.google.com/maps/geo?output=xml&q={0}&key={1}&oe=utf8";
string url = String.Format(ServiceUrl, HttpUtility.UrlEncode(address), MyApiKey);
System.Net.HttpWebRequest request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(url);
request.Method = "GET";
using (System.Net.WebResponse response = request.GetResponse())
{
return response.ToString();
}
}
View 2 Replies
Oct 21, 2010
Let's imaging there are 2 pages on the web site: quick and slow. Requests to slow page are executed for a 1 minute, request to quick 5 seconds.Whole my development career I thought that if 1st started request is slow: he will do a (synchronous) call to DB... wait answer... If during this time request to quick page will be done, this request will be processed while system is waiting for response from DB.[URL] One instance of the HttpApplication class is used to process many requests in its lifetime. However, it can process only one request at a time. Thus, member variables can be used to store per-request data.Does it mean that my original thoughts are wrong?Could you please clarify what they mean? I am pretty sure that thing are as I expect...
View 2 Replies
Feb 19, 2011
I have windows service which need to execute around 10000 schedules (need to send/execute data to all the members).
For one member it's taking 3 to 5 seconds for 10000 schedules it's taking around 10 minutes or so..
But I need to execute all these schedules in one minutes.
View 1 Replies
Feb 21, 2011
I have a method that I call 8 times with different parametres.I use
AvailableYears.AsParallel()
.Select<Int32,DateUsedByThread>(x => GetDataForYearWorker(x,CIF))
.ToList();
GetDataForYearWorker gets the response from a webservice synchronously. It uses very little computing power on my asp.net application, but it ussualy takes 3-5 sec for each webservice response. Because the calls to the webservice are independent of eachother, I want to make tham all at the same time. But it looks like only 2 threads can run at the same time. Why is this and how can I have 8 threads working at the same time?
View 1 Replies
Mar 11, 2011
It seems that by default, ASP.NET 3.5 running on IIS 6.0 does not do any parallel processing whatsoever. With a quad-core system and a test webforms application that runs an infinite while loop on the server, CPU usage never goes higher than 30% regardless of how many clients are connected and independently running the while loop.What are my options for enabling parallel processing?
View 2 Replies
Aug 2, 2010
We are using 6 iFrames on our page. They fetch data from couple of external web services and an internal WCF service and display the data. There is a separate aspx page built for each of the iFrames. From our perf monitoring we found out that at any point only two threads are executing in parallel. Not all 6 threads get executed. What can probably be cause for this? Is there any restriction that more than 2 threads can't be created in parallel? Is there any configuration where I can change this?
View 3 Replies
Sep 24, 2010
I have a static class with a static get property, and in this property, I do this:
// property body
{
// HttpContext.Current is NOT null
Parallel.ForEach(files, file =>
{
// HttpContext.Current is null
var promo = new Promotion();
});
// HttpContext.Current is NOT null
}
This static class doesn't undergone type initialization until a view uses this property.
The problem is that Promotion's static constructor, which is initialized the first time a new Promotion() is created within the Parallel.ForEach(), uses HttpContext.Current. When promo is instantiated within the scope of this Parallel.ForEach(), HttpContext.Current is null, and new Promotion() therefore causes an exception.
HttpContext.Current is not null within the static get property because it's not called until the view uses it (and there is therefore a HttpContext.Current).
If Promotion used HttpContext.Current in its instances instead of its static members, I could probably just pass HttpContext.Current into the new Promotion() constructor:
var context = HttpContext.Current;
Parallel.ForEach(files, file =>
{
var promo = new Promotion(context);
});
But since static members of Promotion need HttpContext.Current, I can't. I could probably redesign the Promotion class to change the static members that need it to be instance members, but they are static for a reason--there would be a large performance penalty if all the members that were static had to be defined instead on each instance each time a new Promotion was instantiated.
What are the possible workarounds for this? I didn't realize HttpContext.Current would be null within the scope of Parallel.ForEach().
View 2 Replies
Mar 14, 2011
I am seeking your expertise in ASP.Net with regards to multi-thread support of WebService in a scenario , that is If a WebService-Client is making simultaneous calls from the same process, the requests will be serialized at the WebServices so that only one-call will execute at any one time , on the contrary, if those calls are sent from different WebService-Clients ( Instances/Processes) , they are processed in-parallel by WebServices.
Have you ever experienced the same with ASP.Net, and what configurations/Settings should be followed, in order for a WebServices to concurrently process simultaneous calls form a single WebClient , when deploying a large number of Web-Clients' instances/processes is impractical in a project-context.
View 1 Replies
Jul 29, 2010
I am experimenting AsyncController feature. What I did is set up two tasks to run in parallel. As in the code below, the problem is that sometime all tasks finished and return successfully, sometime only one task finish and sometime each task finish half of it's work and return. It is weird, what did I do wrong?
[Code]....
View 29 Replies
Jun 24, 2010
I try to get an understanding of the performance of a web page and use Trace.Write
Problem is that I use Parallel.Invoke and starts more threads where I would like to report back using Trace.Write
Parallel.Invoke(() => Chart1AndLegend(param),
() => Chart2(param),
() => Chart3(param),
() => Chart4(param));
I have realized that Trace.Write is not a good way of tracing as it is gives you the time since last entry which makes no sense if more threads are writing.
View 1 Replies
Mar 17, 2011
I've got a WCF web service that runs fine at the moment but there is talk about using it very heavily soon. As part of it's normal process it writes a file out, then reads it back in again (don't ask why, I know it's stupid). I'm concerned that if we start hitting it with lots of requests then the following might happen.
1. Request 1 writes the file out.
2. Request 2 comes in and overwrites the file.
3. Request 1 reads the file back in but this is now the wrong file.
My understanding is that the requests would naturally queue up so that request 2 wouldn't start until request 1 had finished, but I'm not totally sure.
View 1 Replies
Feb 28, 2011
I've got about 30 projects in VS 2008 and we are finally starting to upgrade to 2010. I get change requests pretty regularly so I'm likely to have to make changes to projects that are still in 2008 after VS 2010 is installed. Is it possible to intall VS 2010 without messing up the VS 2008 install? If so is there any particular trick to it so they live side by side or do they automatically do parallel install in different directories?
View 3 Replies
Aug 10, 2010
When I serve an ASP.NET page, can I render the various controls on the page in parallel?
I have a few Telerik controls (RadGrids) on the page and when I step through the page being loaded, it seems as though the controls are databound and rendered serially.
Maybe this behavior is because I am hooked in with the debugger.
Is there anyway to load the page and have select controls build on separate threads? Is that even conceptually possible or must it be done sequentially?
View 2 Replies
Dec 15, 2010
On our webform, I do I remove the vertical lines parallel to each field column?
Visit the webform at [URL]
View snippet code, below:
[Code]....
View 4 Replies
Aug 2, 2010
I have a web app that needs to do two things at the same time. After a user clicks a button on the site, a javascript function needs to execute and while thats working I need the server side code for the button click to execute without waiting on the client side to finish.
To be more specific, the client side function takes about 23 seconds to complete, and while that is running the server side code promts the user for printing a document. So I need the javascript to run "in the background".
View 3 Replies
Nov 5, 2010
I have such a problem. Suppose we have a client (WinForms app), which invokes some methods from webservice. Every 5 minutes (Interval of Timer) I invoke asynchronously webmethod A. Suppose that time of its execution take a 1 minute. When this operation runs, I open some Window, and invokes webmethod B synchronously... or i try to do it. Application suspended, wait until webmethod A ends, then process webmethod B (or I have an exception - Connection was closed, or Timeout...). Is there any possibility to run this A, B webmethods 'parallel'?
View 1 Replies
