How To Scrape An .NET Site That Does All Interaction As Postbacks
Apr 4, 2011
I built a scraper for an ASP.NET site (specifically a Jenzabar course searching portlet) that would create a new session, load the first search page, then simulate a search by posting back the required fields. However, something changed, and I can't figure out what, and now I get HTTP 500 responses to everything. There are no new fields in the browser's POST data that I can see.I would ideally like to figure out how to fix my own scraper, but that is probably difficult to ask about on StackOverflow without including a ton of specific contex
View 4 Replies
Similar Messages:
Jan 18, 2010
I am building a site that need to scrape information from a partner site. Now my scraping code works great with other sites but not this one. It is a regular .html site. My thoughts is that it might be generated some how with php (site is build with php). If it matters here is my code I use. The htmlDocument is htmlAgilityPack but that has nothing to do with it. Result is null on the site I try.
tring result;
var objRequest = System.Net.HttpWebRequest.Create(strUrl);
var objResponse = objRequest.GetResponse();
sing (var sr = new StreamReader(objResponse.GetResponseStream()))
[code]...
View 5 Replies
Aug 17, 2010
I have a button in a repeater control. It saves the data on the screen, so ideally I'd like to do this and not have the screen refresh, as there is no new data to be shown. When I am debugging using localhost, the page just sits there, the data gets saved all is well.However, when I upload the very same page and code to my host, clicking the button causes a full refresh, which as there is a lot of data, means a long wait each time.
View 2 Replies
Feb 25, 2011
how the WCF, DAO and DTO layers communicate with each other? I would appreciate if someone can specify which layer comes first and then how it interacts with the next layer
View 1 Replies
May 18, 2010
I have written this code for opening a text file ,but its saying that file not found.
[Code]....
View 5 Replies
Jun 26, 2010
Basically I have a page where a user can select up to 3 vehicles by make and model. I have 3 divs on the page that is generated with a basic foreach loop. When the user selects a make and model from respective dropdown lists I am firing a javascript function that returns a JSonResult that loads a dynamic partial view into a seperate div. This partial view displays a div with a grid whereby the user can select one of n-number of services using a checkbox.Next what I would like to do is if the user selects a servce for the one of the vehicle selections, I would like to add this selection to a list of "ServiceSelection" that is part of my viewmodel. Ultimately the user would click a button that would redirect to the next step in the process and pass the entire updated viewmodel to this new page.
View 1 Replies
Aug 4, 2010
I want to understand the interaction of telephone with webforms application.
In order to do that I want to create a small application which will be able to receive multiple calls at a time, place them in a queue and then forward calls one by one to a call answerer.
It should also be able to retrieve the Caller Id i.e number from which the call is being made.
How this may be achieved?
If there is any ready made solution available with source code pls provide links.
View 2 Replies
Jan 9, 2010
I am working on a web application project in which i add a dynamic number of LinkButtons at runtime according to user selection, each link should redirect the user to a specific web page ,but what actually happened is that the last hyperlink is the only one responded! the following code show the declaration :
''''''''Public WithEvents ItemName As LinkButton
''Public Sub AddLinkButtons
''''''''For i = FirstIndex To StopFlag
''''''''ItemName = New LinkButton
''''''''ItemName.Text = Dt.Rows(i).Item(1)
''''''''Me.ContentTD.Controls.Add(ItemName)
''''''''Next
''End Sub
''''''''Private Sub ItemName_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles ItemName.Click
''''''''SelectedItemToFind.ItemFlag = 1
''''''''SelectedItemToFind.SelectedItem = sender.Text
''''''''Response.Redirect("Default.aspx?Main=SearchResult")
''''''''End Sub
View 4 Replies
Oct 8, 2010
.net has all these build in GUI elements. The purpose is that we build a .net web application and for the coding itselve I'm very familiar with jQuery in PHP but I noticed that in .Net everybody starts using it too.Is jQuery the best way these days to deal with this in .NET (c#)? or are there others you might recommend?
View 5 Replies
May 31, 2010
I read some article about unitOfwork and repository but i'm still confused about how they interact, and how to use them in the right way.
I'm using an addressbook project to practise on patterns (even if , likely, patterns are not usefull) without any ORM framework for persistence.
My domain objects are (at now) : AddressBook (acts as an application controller), Contact (contains information about each contact in the address book), ContactGroup (mantain collections of contact).
Should i have to use distinct repository object for contact ad contactgroup?
I thought to use a UnitOfWork for the operation about the adding/removing contact to group : the user can add existing contact to a group, create a new contact while adding it to the group or remove contact from group.
View 1 Replies
May 5, 2010
Learning how to do a master page in asp.net. Trying to figure out how my style sheet interacts with respects to the master page and content page. I can get HTML tags like body and the style sheet to react. But when I call the ID attribute of a label no styling takes place. What am I missing here as far as interaction? BTW I'm using VS2008
CSS sample:
body
{
height:1200px;
width:920px;
border-style:solid;
border-color:blue;
padding:10px 10px 10px 10px;
}
#toptext1
{
position:relative;
top:-225px;
right:-500px;
font-size:22px;
font-weight:bold;
}
From the master page:
<body>
<form id="form1" runat="server">
<asp:image id="cookNookLogo" ImageUrl="images/Logo.gif" runat="server"
AlternateText="CookNook" Width="449px"></asp:image>
<p>
<asp:Label ID="toptext1" runat="server" Text="Quality Recipes, Hints and Supplies"></asp:Label>
</p>
From the content page:
<%@ Page Language="C#" MasterPageFile="~/CNMasterPage.master" AutoEventWireup="true" CodeFile="Home.aspx.cs" Inherits="Home" Title="Untitled Page" %>
<asp:Content ID="Content1" ContentPlaceHolderID="head" Runat="Server">
<link href="App_Themes/cn/cn.css" rel="stylesheet" type="text/css" />
</asp:Content>
When I was doing this without a master page it worked so where am I going wrong with the attributes?
View 2 Replies
Dec 11, 2010
I have several ASP.NET sites, hosted on the same domain (different subdomains) and working via HTTPS. I have WCF service, hosted on the same domain. I have a separate state server for sessions.
All sites use cookieless session (if someone asks why - i will reply later, but it's a 'must')
On my sites, I use grid components from Telerik (but actually it does not matter) that ask for data from WCF service and must be filled with this data.
The problems are:
1) I cannot manage to make POST request to WCF service from javascript: for some reason it's always either GET or OPTIONS (no matter if I use Sys.Net.WebServiceProxy invoke or just a plain jquery AJAX request). It happens even if I indicate COOKIE mode, not cookieless. Maybe because of this I am getting "405 Method not allowed" error - WCF is set for POST request, but the site sends GET...
2) I cannot manage to retrieve SESSION from my websites! I tried approach, that I found, but WCF always uses OWN session, instead of 'connecting' to existing ASP.NET session, despite on ASP.NET compatibility mode. I need SESSION to exchange data between my sites and WCF service
3) I don't have idea currently, how to manage security when exchanging data between mentioned ones, in the light of above-mentioned problems...
4) I don't know exactly, how web.config for WCF service should look like in my case, because I've seen tons of different variants, but all for some simple cases.
Would be really appreciated for the help! I'm stuck for 2 or 3 weeks already, far behind the plan, but nothing still works... I can see there are some pieces of info about similar problems, but I'm afraid to miss some small thing - in web.config or method attribute or IIS setting, etc - and f... up everything.
View 3 Replies
Apr 12, 2010
I have 2 nested master pages, lets call them MP1 and MP2. MP1 is top level, MP2 is level
2. Then I have several content pages. Lets say I have 2 content pages for now CP1 and CP2. Each content page has a form that will have some user input and a SaveData()function that saves the data to a database. Each content page also has a "Save" button that calls the SaveData() function.
MP1 and MP2 have several link buttons. The functionality I need is this: Any time a user clicks on a link button in MP1 or MP2, the SaveData() function of the currently loaded page should be called. This is to save the data even if the user forgets to click the "Save" button before leaving the page.
So far I can do this from the link buttons in MP2. I followed the master pages tutorial # 7
In the .cs file for MP2, I declare:
public event EventHandler SaveData;
then i raise a "SaveData" event any time a link button in MP2 is clicked.in every content page (CP1, CP2 etc.) aspx file, I have Master Type directive to assign the Master Page to MP2.
in CP1, CP2 etc .cs file, i have the following code:
protected void Page_PreInit(object sender, EventArgs e)
{
//wire up the SaveData event from MP2 to Master_SaveData function in this page.
Master.SaveData += new EventHandler(Master_SaveData);
}
private void Master_SaveData(object sender, EventArgs e)
{
//when the MP2 SaveData event is raised, save the data in this page.
SaveData();
}
All of this works fine.Problem is I am not able to do a similar thing from MP1.What I have tried so far is to raise an event in MP1 that is handled in MP2. When the event is handled in MP2, it raises the SaveData event to be handled in the page. The event in MP1 is null and hence never raised.
View 5 Replies
Nov 10, 2010
I am building a web application where i am using multilingual support.
I am using variable for label text display.so that administrator can change in one area and that text reflects throughout the application.
which is better less time consuming for display label text?
1)using relational db interaction.
2)constant variable.
3)xml interaction.
how could I found/calculate the processing time of the above three.
View 2 Replies
Jan 18, 2011
i had one .aspx page,which containts in pageload it excutes few stored procesdures for dumping data from Sql Server to mysql Db. So every 30 mints User manulaly running this page.
So i want to automate this page Every 30 mints it automatically should Excutes that page.with out User intaraction.24/7 process.
how to do this. no one want to touch the application.it should be automate..
View 4 Replies
Jan 19, 2011
Firstly there's a great overview of the IIS7 HTTP request lifecycle and various settings that affect performance here:
ASP.NET Thread Usage on IIS 7.0 and 6.0
Very specifically though, in dotNet 4 the defaults for maxConcurrentRequestsPerCPU and requestsQueueLimit are set to 5000. E.g. equivalent to: (in aspnet.config):[code]....
Seems to me that on a multi-CPU/core server the requestQueueLimit here will always be invoked well berfore the 'perCPU' limit. Thus, if a max of 5000 requests per CPU is what you actually want then I would expect that the requestQueueLimit needs to be increased to 5000 * CPUCount or just disabled altogether.
Is my interpretation correct? If so can I disable requestQueueLimit? (set it to zero?). The documentation on this setting doesn't appear to address this question (so maybe I'm missing something or misreading?)
** side note from the above article: The requestQueueLimit is poorly named. It actually limits the maximum number of requests that can be serviced by ASP.NET concurrently. This includes both requests that are queued and requests that are executing. If the "Requests Current" performance counter exceeds requestQueueLimit, new incoming requests will be rejected with a 503 status code)
View 4 Replies
Jul 20, 2010
This is kind of cross-post; however, can a custom http handler be automated to run either using a timer or a filesystemwatcher to post an xml document to another server (with no human interaction)?
View 2 Replies
Mar 12, 2010
I have got two pages - a master pages (Master1) and an .aspx page (default), with the content page attached to the master page.
However, I have run into a problem
I have some data controls like drop down boxes and textboxes in the master page, with a gridview of data in the content page. What I want to achieve is a situation where when a user searches for an item using the texbox in the master page, the gridview in the content page displays the results of that search parameter made.
View 1 Replies
Feb 16, 2011
I would love to create a "daemon" which in specified time intervals will move-transfer any zip files uploaded by the users from a.example.com to b.example.com
From the info i gathered so far,
The daemon will be an .ashx generic handler. The daemon will be triggered at the specified time intervals via a plesk cron job The daemon (thanks to SLaks) will consist of two FtpWebRequest's (One for reading and one for writing).
So the question is how could i implement step 3?
Do i have to read into to a memory() array the whole file and try to write that in b.example.com ? How could i write the info i read to b.example.com? Could i perform reading and writing of the file at the same time?
No i am not asking for the full code, i just can figure out, how could i perform reading and writing on the fly, without user interaction.
I mean i could download the file locally from a.example.com and upload it at b.example.com but that is not the point.
View 8 Replies
Nov 1, 2010
I am using IIS Url rewrite to convert urls from /mypage.aspx?q=value to /page/value.html.
Everything works fine during the first call to the page but on postbacks the url is transformed into
/page/value.html?q=value
when using asp controls such as GridViews, Pagers or Custom Controls.
As a result the Server returns a 404 Error and i can see the corrupted url on the browser's address bar.
This problem was observed when i wanted to change the page of a gridview using a pager.
View 4 Replies
Oct 7, 2010
have a javascript function that fires by clicking any element on the asp.net webpage.When it fires, we have three scenarios:1- full postback (when clicking on a button outside the update panel)2- Partial postback (when clicking on a button inside the update panel)3- No postback (when clicking on a table cell for instance)I was able to detect when a partial postback occurs by using the async property demonstratedbelow, but I could not find a way to differentiate between a full postback and a no
View 5 Replies
Oct 18, 2010
I have a peculiar situation at hand in which I need to validate a pair of dates in an ASP.Net web app.This is how my page looks like : I have a "From Date" field which consists a set of three dropdownlists ( for Year , Month and Date) I have a "To Date" field which also consists a set of three dropdownlists ( for Year , Month and Date). There is a "Download Report" button on the screen ( which allows a user to download a report for the specified date range) All these six dropdownlists are wrapped in a user control. Now all the 6 dropdownlists have a default "please select" value. When a user lands on the page for the first time , all dropdownlists are set to "please select". The "Download" button is enabled only after valid 'From' and 'To' dates are selected. Now my requirement is that :
When a user lands on the page for the first time , no attempt to validate the dates should be made until he selects all the 6 dropdownlists at least once. When the 6th dropdownlist is selected , the From and To dates should be validated. Once a user has selected all 6 dropdowns once , subsequently on every dropdownlist postback , the dates should be validated. My Issue: What's the best way to track these 6 distinct postbacks ( for each of the dropdowns) before starting to validate the From and To Dates.
I could maintain a flagin ViewState .. but is there a better way?
View 1 Replies
Feb 3, 2011
Our solution is built on ASP.NET v1.1 framework. User is presented with login form, so upon providing credentials this information is posted back to the server. Upon postback SessionID changes and our application crashes as some information which is required for processing is stored in the cache and is identified by SessionID string as a part of the name.
This SessionID change happens absolutely randomly and only to some of the clients. Most of the time browser used to view the page and post info is IE8. I cannot reproduce this issue in our test environment, as SessionID is persistent though-out the whole testing process.
I've already checked all solutions, i.e.
Session cache is used on Page_Load to retrieve/store some information, so it's initialized and contains data stored. I've made a Health Monitoring check on Application_End event to capture any possible AppDomain crashes using Reflection and Diagnostics Libraries and retrieving ShutDownMessage from httpruntime object, but that's also not a case. Cookieless attribute of the sessionState in web.config is set to false (using URL to store SessionID is not an option) All MS security and bug fixes are installed on the server. IIS Server settings are similar to the ones we use in Test Environment.
View 2 Replies
Oct 7, 2010
On an ASP.NET page with a tabstrip, I'm using the hash code in the URL to keep track of what tab I'm on (using the BBQ jQuery plugin). For example:
http://mysite.com/foo/home#tab=budget
Unfortunately, I've just realized that there are a couple of places on the page where I'm using an old-fashioned ASP.NET postback to do stuff, and when the postback is complete, the hash is gone:
http://mysite.com/foo/home
... so I'm whisked away to a different tab. No good.
This is a webforms site (not MVC) using .NET 4.0. As you can see, though, I am using URL routing.
Is there a way to tell ASP.NET to keep the hash in the URL following a postback?
View 1 Replies
Nov 19, 2010
I am having a custom ASPX page with following structure. Its having 2 update panel and I have a ImageButton outside them. The two UpdatPanel are triggered by this ImageButton.The Problem is whenever user clicks the Imagebutton, the image in the ImageButton is getting reloaded...it disappears and the reappears. I am not able to figure out why the postback is happening when the imagebutton is place outside UpdatePanel. Is it because I am using this Imagebutton for triggering AsyncPostBack in UpdatePanel?
[Code]....
View 6 Replies
