I want to write crawler for screen scrapping What I want is, I want to get price of particular hotel from a website, like here is website e.g. In the above URL, there is list of hotels and its price. I want to get the price of the beaufort
View 3 RepliesI am working on Screen Scrapping, I was able to do, but some of the website have captcha and need to enter captcha information to proceed further
Is there anyway to read captcha information and submit those values that are in captcha or how can we handle this scenario?
I need to get a share price of a company in stock market from this site, [URL] I need to show it in my Webpage in a Webpart
View 1 RepliesNow it seems like a really simple question and I may just be being thick, but what I'm trying to achieve is basically print an array to screen in the following format: Item 1, Item 2, Item 3, Item 4
Although I say to screen, as that was the best way I could describe it, I'm actually writing it to the page inside some Javascript.The way I'm currently going about writing it out is as follows:
for each b in theDates
Response.Write("'" + b.CallDate + "',")
next
But obviously that returns a string of Item 1, Item 2, Item 3, Item 4,.Is there a simple way of getting rid of the last comma or am I going about this completely wrong?
I have develop lots of web aplication including web-site with paypal integration and now I want to develop scrapping in asp.net. Any one has knowledge about what is scrapping and how it is implemented in asp.net. I am new to this scrapping.
View 2 RepliesI implemented : <%@ OutputCache Duration="43200" VaryByParam="none" location="Server" VaryByCustom="RawURL" %>
I have got a sitemap.xml with all url(about 12000) possible in my site.
I would now if it's judicious to create an application that will parse my sitemap in
order to request all url?
The Goal is caching all my web site to increase velocity.
I precise my question :Each page take about 10 sec to be created and the cache duration and cache duration is 12 hours
I have built a simple threadpool based web crawler within my web application. Its job is to crawl its own application space and build a Lucene index of every valid web page and their meta content. Here's the problem. When I run the crawler from a debug server instance of Visual Studio Express, and provide the starting instance as the IIS url, it works fine. However, when I do not provide the IIS instance and it takes its own url to start the crawl process(ie. crawling its own domain space), I get hit by operation timed out exception on the Webresponse statement. Could someone please guide me into what I should or should not be doing here? Here is my code for fetching the page. It is executed in the multithreaded environment.
private static string GetWebText(string url)
{
string htmlText = "";
[code]...
I would like to state literally that I really do not want to reinvent the wheel, but as you know some times we get some unique "Must-requirement(s)" that will hinder us to re-use the open source codes.I want a proper, flawless and consistent Web Crawler. Basically, I want this Crawler (As a Web app NOT desktop app - Of course based on asp.net and C#) to grab the pages of any website and store them locally (Including resources like images and CSS...etc), download them locally and adjust any resource hyperlinks to point to the locally downloaded resources.
I saw HTTrack (http://www.httrack.com/), and it seems quite excellent, but the problem is that I want this Crawler to be a part of a system which includes other features and process. So, I really can't have this Crawler as an external tool.Main challenges:1) User should be able to specify to which level s/he wants to crawl, which means: User might specify a sub-site and want to crawl everything underneath it and not the upper level. You see ? So, there should be full crawling for the entire site and partial crawling.
2) URLs and how to deal with them? I faced some weird URLs and it was hard for me to identify the actual page because there is no file name. How to handle that? For example: (http://www.blue1.com/en/uk/Travel-info/At-the-airport/Security-control/) this is a URL of a website that is built on EPiServer (.Net based) but as can be seen from the URL that there is not actual aspx page, Therefore, how to deal with such URLs ?I have already started developing a POC using HttpWebRequest class, but frankly I am totally dissatisfied with it. It is inconsistent and the generated static content misses a lot of images and styles. Besides, the threads act up sometimes strangely.I would greatly and sincerely appreciate any input (Approaches, source codes, ideas , links...etc)P.S. I already saw: (http://www.codeproject.com/KB/IP/Crawler.aspx) and (http://www.codeproject.com/KB/aspnet/ZetaWebSpider.aspx).
I have question regarding printer option page which pop up when i click on print button on crystalreportviewer and when i click okay on that pop up page it displays print preview of the report.
Is there any way to skip that pop up window(Printer Option) and just display (PDF)print preview screen instead?
(I am not exporting report to pdf, just want to display printview screen and when the user click on print button it will print out the page)
I have tried following code but i am getting error:
"Object reference not set to an instance of an object."
[code]....
I have upgraded web application from VS.2008 to vs 2010. I was using previously ReportViewer 2008 control in a page and replaced with 2010 ReportViewer Control.
I have installed the ReportViewer Redistributable also in my local pc as well as in the server. Also as per the new requirement I have placed the Scriptmanager in the web page where the reportviewer control located.
While running the page, the reportviewer run the report and is not showing the data on the screen. It shows the blank screen. But If I export the data to execl or world it is showing the data. Also the page numbers in the reportviewer showing the total pages available in the navigation bar. I am sure the report is running and bringing the data to the screen but not showing it.
The below is the exact scenario, One form will be there and when user click on "Edit" hyper link other window will open to get more details. In that new window also I have a grid or a table which has other hyperlink which lead to open other window and in that window I will enter few details and save it....
[code]....
I have created my first MVC application. I am using forms authenticataion. I have created a custom action filter MyAppAuthorizationFilter which inherits from AuthorizeFilter. Inside this filter I'm authenticating the user credentials by checking them against a database table (by overriding OnAuthorize method). If the user is not authenticated, I am redirecting the users to a login page (which is view of LoginController). I am using this filter agaginst my main controller so that all actions in this filter will execute only if the user is authenticated.
I do not have any problem when I use the asp.net development server (shipped with Visual Studio). But when I deploy the app to our IIS7 server, I can see the login screen when I access the default url, but after entering the credentials (both correct and incorrect) and posting back the Login page, it stays on the Login page (clears the pwd, but not the username). Its never getting redirected to the home screen or invalid login screen.
IIS 7 server has .net framework 3.5SP1, but no MVC. So I deployed the System.Web.Mvc dll in the bin folder.
Here's my Code (I've omitted the views and models as they are not needed here):
[MyAppAuthorizationFilter]
public class MainController : Controller
{
//
// GET: / [code]......
I am using the below code where I put this tag on a .aspx page:
"<a name="tips"></a>"
Now if I from another page run the code in the hyperlink to navigate to this page2 where I have the above tag. The page will scroll down to this tag automatically where this tag will be at the TOP of the screen.
My question now is if it is instead possible to scroll to the position where you have this tag at the BOTTOM of the screen instead ?
[Code]....
Is it possible to take a screen capture of a particular area of a web-page from the web-page's own ASP.NET code?
I need to display 6 images that are stacked on top of one another using CSS, and I'd like to offer the user the ability to right-click save-as if possible by stacking the screen capture on top top of the original image stack.
how i can open my asp.net program on full screen (like i press F11)
Through C# code ?
I have a textbox with textmode :password.
But after the postback the value disapears.
How can i retrieve this string value as simple as possible together with a very basic encryption?
Is there a way to generate reports using asp.net?I am using an oracle database for datastore Also using crystal reports to generate reports from database now.
View 3 RepliesI have a simple page with a Grid that I'm binding a collection of objects to. I also have some simple functionality on the grid to edit and save rows. I would like to write unit tests for this page, but it's not really making sense to me.For example:
Private Sub LoadGrid()
'Populate Collection
grid.datasource = MyCollection
grid.databind()
end sub
I guess a Sub really doesn't need a unit test, but what if this were a function that returned true when the grid had been loaded. How do you write a unit test for this? What other test should be done on a simple web page like this?
I want to show my data on an ASP.NET page using C# in XML format
<person>
<email>a@a.com</email>
<dob>YYYY-MM-DD- HH:MM:SS</dob>
<city>XYZ</city>
</email>
</person>
Do you have any code with examples.
I have a page with a gridview on it. It selects rows relevent by using a querystring id passed on from the previous page.
From this gridview, i want to write only the displayed rows to a .txt file. How can i do this?
Is there a way to read and write files on a client's computer using a web based application? Basically I need to create a internal app that will take two input CSV files and convert it into an output CSV file that is written to the same location as the input files on the client's computers.
Since this is an internal app, I don't have to worry about security. I can easily write a C# app for this but I want to create a ASP.NET app (or any other format). I even looked into creating a HTA application.
I am trying to write some lines at runtime to a text file using streamwriter, but i have to allow only 100 lines to be written to that file, if 101 line is supposed to be written to that file than first line of the file must get discarded/deleted and file must allow writing the 101 line so the file will actually always contain only 100 lines.
View 10 RepliesIs there a way to write xml directly to an object that can represent it in memory and use that with XPath instead of an .xml file?
View 1 RepliesI want to write an .xml file using the following code into the App_Data/posts, why is it causing an error:
Stream writer = new FileStream("..''App_Data''posts''" + new Guid(post_ID.ToString()).ToString() + ".xml", FileMode.Create);
I'm interested writing a set of custom controls (most likely in ASP.NET), was wondering what sort of guidelines I should follow.- Is there any generic requirements for UI controls that should be followed?
- When writing documentation / samples etc, any guidelines on what they should include?I know it's very general question but interested in your feedback on how I should I begin and maybe have a set of tight guidelines/framework from the beginning.