I need to screen scrape a web page and change its style to match the look and feel of the site where it will be displayed in. Is this possible? I'll be using asp.net to do the screen scraping.
View 1 Replies
Every night I have a program that runs that creates a natural gas report. In this report is something called "expert analysis." The is just text which is a commentary of the current Natural Gas market and because it's a volatile market there is new commentary every day. So also every day, an employee in our office runs a public webpage which displays this text, copies and pastes it into a webpage that is part of our system and it gets saved to a file which gets pulled into our nightly report.
The page that is in our system is as ASP page and it uses FileUp to create the file. We want to break our dependancy on FileUp (it's a SoftArtisans product, an excellent vendor, but we are trying to save some money). Right now we don't have FileUp because our trial expired. So every day I run the public webpage and copy the text and paste it to my expert.txt file all manually. Another problem is that the expert analysis is updated after our office staff has all gone home,
Here is my question - can I write a program in .NET that runs a page, parses the resulting HTML and extracts out the expert analysis all automatically? I won't need FileUp anymore becaus I can use the .NET upload control. And I won't have to log on every night to manually copy and paste the expert analysis. Is that what web/screen scraping is? What happens then the public page's format changes because they do maintenance on their site? Is it just like if I were to run [URL] and try to read the Moderator column of the ASP.NET forum? I'd be okay for a while (would I?) but then if the layout changed I wouldn't find the Moderator anymore?
I've created a web application in asp.net so far. where i've tried to get some data(site scraping) from secure page of a web site.I've used the HttpWebRequest class for this functionality but i haven't accessed the secure page yet. Every time the login pages was scraped not secure page.I have the site user id and password and don't know that which language site has been developed in.
View 1 RepliesI'm trying to write a small application to collect(Scrape) one piece of data from a web site. I would like to be able to simply run the app and it will open the page, find the one piece of data and display it. So far so good...my problem is that the web site is a secure site, meaning I have to provide a user name and password. I've searched all over the web, found many discussions but have yet to find anything that provides specifics on how to accomplish this. I understand a little bit about tokens etc, but I'm really looking for a detailed description of how to do this. Please feel free to direct me to a different forum if I'm in the wrong place.
View 3 RepliesI am building a site that need to scrape information from a partner site. Now my scraping code works great with other sites but not this one. It is a regular .html site. My thoughts is that it might be generated some how with php (site is build with php).
If it matters here is my code I use. The htmlDocument is htmlAgilityPack but that has nothing to do with it. Result is null on the site I try.
[Code]....
this is from the w3 validator, might have something with this? The site checked is this
[URL]
I am unable to validate this document because on line 422 it contained one or more bytes that I cannot interpret as utf-8 (in other words, the bytes found are not valid values in the specified Character Encoding). check both the content of the file and the character encoding indication.
The error was: utf8 "xA9" does not map to Unicode""
I've created a web application in asp.net so far. where I've tried to get some data(site scraping) from secure page of a web site.I've used the HttpWebRequest class for this functionality but I haven't accessed the secure page yet. Every time
the login pages was scraped not secure page.I have the site user id and password and don't know that which language site has been developed in.
I am new to web programming. I am developing a web program using asp.net(vb) that scrapes data of a certain website. I am using System.Net.HttpWebRequest and System.Net.HttpWebResponse to read the HTTP codes.My problem is I can not retrieve the codes of certain frame/container where the data that I needed is located in the website.I understand that iframe has its own URL or link aside from the main URL of the website
View 1 RepliesI seem to be having some challenges with the data I am retriveing from a Webpage using the Webclient class. The code works fine, however I observe that the regular expression is not picking up the negative or positive sign in the Daily_Movement data. For example, a daily movement can be -0.31 or +0.31 but the code is not picking the sign in front of the decimal values.Here is my code
[Code]....
I think where the problem lies is the part of the code Regex r1 = new Regex("<span class="quoteData">.*</span>"); It picks up the values between the tag quite well, but not the signs in front of it. [Code]....
this may or may not be a big deal but what I am looking to do is based on some data returned to my view I want to change the background-color to what value is being returned. Black, Yellow, etc.... How can I in my aspx page set my Style background-color based on this value in the model. I know Attributes are used in Web forms control but not sure in MVC how to do this
View 3 RepliesI want to change each items color in the asp menu control while the menu is generating dynamically from a database.
i.e. 1st item's background color is yellow and 2nd item's background color is red and so on.
In Visual Studio 2010, is there a way of changing the Cascading Style sheet on the server-side based on the client browser? I want to create different style sheets for the different browsers (Firefox, Chrome, IE, Opera, Safari etc..) and specify which one to use from the master page.
View 2 RepliesI need to change the background color of each menu item of asp menu control while the menu items are populate dynamically using a database. i.e. 1st menu item in black background, 2nd in green and 3rd in red and so on. If you can explain using C# it is great.
View 2 RepliesI've just created a Menu navigation that's linked to a SiteMapDataSource (which is in turn getting data from a .sitemap file that I've created). It looks great and acts accordingly, but my only question regarding this is there any way to just change the style (font color, in this instance) of just one element in the menu item? I just want it to stick out a little more than the rest of the menu items.
View 5 RepliesI would like to change color of the gridView depending on the value from table that is bounded to the grid. I actually don't show this value on the grid. I tried by using additional field with visible=false, but the cell is string empty in this way. I don't know how to solve this in some other way?
I used this code, but as I said the value is empty:
[Code]....
Is it possible to solve this by getting value directly from bounded table and not from rows cell?
I have a trial version of VS2010 and created a simple out of the box ASP .Net web application. I have been playing around with the css file for the project and have yet to figure out how to effect the menu tabs.
View 2 RepliesIn my webform I have a datalist that displays images that are approximately 200px each. The datalist is set to RepeatColumns = 5, RepeatDirection = Horizontal (display 5 images per row). With the padding and spacing, each datalist row takes up about 1200px in width.
The problem is that if a user with a 1024x768 or 800x600 resolution visits the page, he'll see the horizontal scrollbars. It's not really a problem, but, well...
Is there any way I can call a javascript function in page_load that will get the screen dimensions, and send that javascript result to my serverside so that I can change Datalist.RepeatColumns?
My javascript function would probably use screen.width and screen.height, but how do I send those values to my codebehind?
For reasons, I have an <a> tag in my master sheet rather than an ASP link. I want to access this html control from the server when a post back happens and change its style sheet. I have tried many a thing but as yet am still unsuccessful.
in short this is what i can't do but want to...
page.form.controls.item(0).sytle = "bob"
..but can't.
I want regex operation for removing style tags,css,scripts and html tags from html to plain text in asp.net c#...
View 1 RepliesI have a Master Page with css style sheet and Content page. i am using IE 8 and VS 2008. I have a content page with a textbox and a submit button, however the value of the textbox is not being captured when the user clicks the submit button in the code behind file(cs file). However it is working in Firefox and Chrome when I display the value in a message box, however not in IE .
Below is most of my Master Page
[Code]....
Below is my Code for my content page
[Code]....
In my code behind for the submit button is
[Code]....
None of the message box is showing the values using IE but the same messagebox is showing values in Firefox and Chrome.
I am generating the controls for an MVC Create Form using html.editorfor().How can i add style properties to whatever is created? For instance i have a control which is generated by html.editorfor() as a single line textbox but i want it to be a multiline textbox.
View 2 RepliesI'm building a html table dynamically in an ASP.NET code behind file using C#. I basically loop through a set of data which is an unknown number of records and split a string containing all the values to make the required number of tds. I display the html by assigning it to an asp:Literal control. However I can't get the table to fit the screen - the browser is adding a horizontal scroll bar and the full table is well off the screen. I tried in IE 8 and FF 3.6.13. Most things I've read online about it say to set the width to 100%. I'm doing this but it's having no effect.
<div id="paging">
<asp:Button ID="btnPrev" runat="server" Text="Prev" CssClass="niceInput" onclick="btnPrev_Click" Enabled="False" />
<asp:Button ID="btnNext" runat="server" Text="Next" CssClass="niceInput" onclick="btnNext_Click" Enabled="False" />
[code]...
how do i create website which supports various screen resolution & Multi browser support (i.e 1024*768 ) and others using asp.net...
i have developed website in asp.net (vb) my monitor resoultion is 1024*768 but when i try to access my site in other monitor having the resolution more than 1024*768 then the controls are unmanaged .... in the webpage...
how to provide button style to an html actionlink
View 1 RepliesI am new to Asp.net and i saw many tutorials talking about css and Html and Xml Style sheet so please can anyone tell me what is the difference between those 3 Languages and what they are used for ? Also i want to know if we need to know HTML since when i was training on visual studio i saw that it is automatically generated when we add a control from the tool box .
View 4 RepliesHow to insert extenal style sheet into VB?
Without inside the <head></head> tag.
Because i have link with master page, so don't have head.
