Web Forms :: HTML Page Scraping - Picks Up The Values Between The Tag But Not The Signs In Front Of It?
Jul 29, 2010
I seem to be having some challenges with the data I am retriveing from a Webpage using the Webclient class. The code works fine, however I observe that the regular expression is not picking up the negative or positive sign in the Daily_Movement data. For example, a daily movement can be -0.31 or +0.31 but the code is not picking the sign in front of the decimal values.Here is my code
[Code]....
I think where the problem lies is the part of the code Regex r1 = new Regex("<span class="quoteData">.*</span>"); It picks up the values between the tag quite well, but not the signs in front of it. [Code]....
I just come across this W3C validation errors: I have a LinkButton1 on my .aspx file, and in my c# code I want to disable this LinkButton1 in certiain circumstance, hence i was using this code:
I need to screen scrape a web page and change its style to match the look and feel of the site where it will be displayed in. Is this possible? I'll be using asp.net to do the screen scraping.
Every night I have a program that runs that creates a natural gas report. In this report is something called "expert analysis." The is just text which is a commentary of the current Natural Gas market and because it's a volatile market there is new commentary every day. So also every day, an employee in our office runs a public webpage which displays this text, copies and pastes it into a webpage that is part of our system and it gets saved to a file which gets pulled into our nightly report.
The page that is in our system is as ASP page and it uses FileUp to create the file. We want to break our dependancy on FileUp (it's a SoftArtisans product, an excellent vendor, but we are trying to save some money). Right now we don't have FileUp because our trial expired. So every day I run the public webpage and copy the text and paste it to my expert.txt file all manually. Another problem is that the expert analysis is updated after our office staff has all gone home,
Here is my question - can I write a program in .NET that runs a page, parses the resulting HTML and extracts out the expert analysis all automatically? I won't need FileUp anymore becaus I can use the .NET upload control. And I won't have to log on every night to manually copy and paste the expert analysis. Is that what web/screen scraping is? What happens then the public page's format changes because they do maintenance on their site? Is it just like if I were to run [URL] and try to read the Moderator column of the ASP.NET forum? I'd be okay for a while (would I?) but then if the layout changed I wouldn't find the Moderator anymore?
I am trying to get these three fields on the screen while user enters I am retreving the user enter data on front end.. when I am debugging I am not seeing these fields..
On the view I am using beginForm
<% using (Html.BeginForm("Update", "Home", FormMethod.Post, new { @id = "id" })) { %>
my method..
public JsonResult Update(StudentInfo info) { ///Update return Json(Status.ToString()); }
when I see in info I am not getting these three fields..
I am using asp.net 3.5 c#.If i want the application (user) to pick multiple things from a dropdownlist, view them and then insert into database.. how can i do that?Is it possible to make it so that once he picks something from the dropdownlist it is added to a list (or a table) ,, so he can check them before inserting to DB??
I want to insert a particular value against a range of dates into my sql server database from the front end using c#.
the table has the column date and worker. so i desire that the user writes the name of the worker and then enters a From and To range into two textboxes so that thorugh a loop function the name of the worker gets written into the table along with the range of chosen dates.
I have an ASP.NET website in which the user has the Login/Logout functionality. There are some pages which are viewable only if the user is logged in. Now, after the user signs out, I noticed that the user is still able to see this restricted page on pressing the back button. I want to create the functionality such that once the user signs out, the page should not be viewable. How can that be done?
P.S I am not using built in Login controls in ASP.NET
I need to code a e-commerce site but about the front end I thought about building a "flowing" menu, how it works is, in the top row you have "Retail" and "Wholesale", if I move my mouse over the retail word I want in the next row, data to be loaded like the main categories if you clicked a category word, then in the 3rd row you get the sub categories then you can click on a subcategory name and then a 4rth row will appear with more subcategories, and so on and on, the reason for this is that the business has alot of sub categories going very deep down,
I was wondering how to create a front page news function with C# as the language.
Basically I want to be able to use a one of my forums in the category "Site Discussion" called "News" as a way to connect with the front page (homepage) of the site to display a few of the newest news articles each with a title, image, and a short blurb of what the post is about. Also I want them to have a link that connects to the news post in the forums.
***NOTE*** I am using YAF (Yet Another Forum) for my forums.
An example of this from another site is [URL]
As you can see in the above site in the "Today's Top Stories" section it shows 4 news posts (1 main, and 3 other top stories) and if you click on them it links to the news post (which is also accessible via the forums in the news section).
This is basically what I want to incorporate into my site, but using the YAF forums, note an admin console is also needed to do this, I have a serperate admin console that I will set-up for it basically that's when you write it and from there it displays on the front page and in the forums.
I have a Java Script popup messae box that is displayed when an application user first displays the web page. Everything works great, but when the popup displays the background of the screen is white, with the popup in front of it. Instead I would like for the popup to be displayed in front of the web page after it has been rendered with all of the controls on it.
<%@ Page Language="C#" codebehind="Default.aspx.cs" inherits="Colso.Site.Default" masterpagefile="~/Master.Master"> <script runat="server"> protected void Page_Load(object sender, EventArgs e) { StringBuilder sb = new StringBuilder(); sb.Append("<script language='JavaScript'>"); sb.Append("alert('message to client')"); sb.Append("<"); sb.Append("/script>"); ClientScript.RegisterStartupScript(this.GetType(), "PopupSb", sb.ToString(), false); } </script> <asp:Content id="contentPage" runat="server" contentPlaceHolderRoot"> // Controls not shown because I don't want too many lines </asp:Content>
I have written a ajax function that I will check the result of a div tag to see if I want to hide the partial view and show the result. The result depends on what is coming back from the controller and the html(result variable). It should only go to the else statement if the return html string does not contain a div tag with id "valRegno". I do not know how I can find the div tag and check if it is there or not. The "result" htmls are nice and show me the details I am intrested but I can not extract it from the html string.
I am building a site that need to scrape information from a partner site. Now my scraping code works great with other sites but not this one. It is a regular .html site. My thoughts is that it might be generated some how with php (site is build with php).
If it matters here is my code I use. The htmlDocument is htmlAgilityPack but that has nothing to do with it. Result is null on the site I try.
[Code]....
this is from the w3 validator, might have something with this? The site checked is this
[URL]
I am unable to validate this document because on line 422 it contained one or more bytes that I cannot interpret as utf-8 (in other words, the bytes found are not valid values in the specified Character Encoding). check both the content of the file and the character encoding indication.
The error was: utf8 "xA9" does not map to Unicode""
I've created a web application in asp.net so far. where I've tried to get some data(site scraping) from secure page of a web site.I've used the HttpWebRequest class for this functionality but I haven't accessed the secure page yet. Every time the login pages was scraped not secure page.I have the site user id and password and don't know that which language site has been developed in.
I have a table in sql having a numeric data type column,Now i want that whenever user left this field blank from front end aspx page, null value should be inserted in this column from stored proc.How can i achieve this? Also what kind of data type is most appropriate on front end for such taking kind of value?
I am working on a project that uses data scraping technique to retrieve some url links. I encounter this problem when i pass in the url of a [previous page button] link frm the html code and pass it in to httpWebRequest, the httpWebResponse that i get back is different form the actually content. i have been try to sovle this problem for days and no result, as anyone encounter similar problem and manage to sovle it? below is my sample code: [previous page button] [URL] note: i have change the domain name to a dummy address which is localhost
I think I need to drop in some escape characters, but I'm not quite sure where. Here is the javascript function I'm attempting to call:
function setData(associateValue, reviewDateValue) { var associate = document.getElementById("Associate"); var reviewDate = document.getElementById("ReviewDate"); associate.value = associateValue; reviewDate.value = reviewDateValue; }
Here is the asp .net mvc line where I'm attempting to create a Radio button with a click event that calls the above function and passes data from the model as javascript parameter values.
<%= Html.RadioButton("Selected", item.Selected, new { onClick="setData('<%=item.Associate%>','<%=item.ReviewDate%>' )" } )%>
The above throws a bunch of compile issues and doesn't work. A call such as the following does call the javascript, but doesn't get the data from the model.
... nothing is displayed. However, outside of the <% %> tags (ie in the HTML) £ displays correctly.
I have no trouble displaying the usual alphanumerics, it's just the £ sign that is proving problematic. The underlying file is in Windows 1252 encoding, and I need to serve it as such. If I save the file as UTF-8, I get mojibake instead of a £.
what I can do to make this work, or any settings that might be preventing it from working (other than saving the file in a different format)?
