Can't Scrape A Site - Result Is Null On The Site
Jan 18, 2010
I am building a site that need to scrape information from a partner site. Now my scraping code works great with other sites but not this one. It is a regular .html site. My thoughts is that it might be generated some how with php (site is build with php). If it matters here is my code I use. The htmlDocument is htmlAgilityPack but that has nothing to do with it. Result is null on the site I try.
tring result;
var objRequest = System.Net.HttpWebRequest.Create(strUrl);
var objResponse = objRequest.GetResponse();
sing (var sr = new StreamReader(objResponse.GetResponseStream()))
[code]...
View 5 Replies
Similar Messages:
Apr 4, 2011
I built a scraper for an ASP.NET site (specifically a Jenzabar course searching portlet) that would create a new session, load the first search page, then simulate a search by posting back the required fields. However, something changed, and I can't figure out what, and now I get HTTP 500 responses to everything. There are no new fields in the browser's POST data that I can see.I would ideally like to figure out how to fix my own scraper, but that is probably difficult to ask about on StackOverflow without including a ton of specific contex
View 4 Replies
Jan 26, 2011
I have a requirement where I have to transfer a user from Web Site 1 on Server A to web Site 2 on Server B.
On Web Site 1, I have to provide controls to enter user id and password and which have to be validated on Web Site 2 on server B, after validating them I have to redirect the user to Web site 2.
what is best way fo doing it. code examples are greatly appreciated.
Note: On Web Site 2, user login functionality is already existing and it is provided by ASP.Net login control, am not sure how to handle the user login process from two different places.
or is it the best way to move Web Site 2 from Server B to Server A so that, the same login controls will be shown using Iframe on Web Site A.
View 6 Replies
Jan 18, 2010
I am building a site that need to scrape information from a partner site. Now my scraping code works great with other sites but not this one. It is a regular .html site. My thoughts is that it might be generated some how with php (site is build with php).
If it matters here is my code I use. The htmlDocument is htmlAgilityPack but that has nothing to do with it. Result is null on the site I try.
[Code]....
this is from the w3 validator, might have something with this? The site checked is this
[URL]
I am unable to validate this document because on line 422 it contained one or more bytes that I cannot interpret as utf-8 (in other words, the bytes found are not valid values in the specified Character Encoding). check both the content of the file and the character encoding indication.
The error was: utf8 "xA9" does not map to Unicode""
View 4 Replies
Nov 23, 2010
I've been using an asp 3.0 widget for this purpose , but it been broken for a while and rather than get it back up, I'd love o get a native asp.net solution, can any one point me in the right direction pleaseI once wrote some asp.net code to do this, but the permissions on the server, which I found to be common every where prevented asp.net native picture upload, perhaps i got it wrong, well now I wanna fix itI need the code to work in a shared hosting IIS 7 environment. I use vb.net, but can usually convert c# to vb.net
View 1 Replies
Oct 21, 2010
Using Visual Web Developer 2010 Express. I was used to creating a web app anytime and I would get a default page for starters and be happy. But today I find that when I pick either the ASP.NET Web Site or ASP.NET Web Application templates I get site.master and a bunch of site admin files that I don't want. Alternately when I select the empty versions of those templates, I get almost nothing at all. An almost empty web.config page and no Default.aspx page. There is nothing magical about having a default.aspx page in place, but it is an indicator of a change when I can only get a virtually completely empty website or a website with a bunch of bells and whistles I don't want.
Also, for some reason, when I create a new app or website as just noted, in the Solution Explorer I get aspx.designer.cs files showing as well as web.config files for both debug and release. Somehow some settings and templates must have changed, or am I missing something here? How can I get my settings/templates to go back to the way they were?
View 1 Replies
Feb 2, 2010
I am having a little difficulty trying to wrap my head around site structures. I have quite a large asp.net site, for which: Admins can see a user list, Each user can have many accounts, and for each account can have many items. I can picture this as a breadcrumb trail of editing a particular item. User List > Mr Bob > Accounts > BOB77 > Items > Item32 > Edit
User List = All the users Mr Bob = A user the administrator has selected from the User List Accounts = A list of the user's accounts 12BOB = The administrator has selected the account named 12BOB Items = A list of the items an account contains Item32 = The item that the administrator selected Edit = The action that the administrator wants to do. I can picture how this would look like if it was using ASP.NET MVC with the URL, but I am unsure how to map this out using Webforms, and in the physical filesystem.
This is what I have thought up about how I am guessing the physical structure would look like. Will this have to use session variables to achieve what I am trying to do?
/Users/User/Edit.aspx <- for editing a user
/Users/User/View.aspx <- for viewing a user
/Users/User/Accounts/Default.aspx <- for viewing all accounts
/Users/User/Accounts/Account/View.aspx <- for viewing an account
/Users/User/Accounts/Account/Edit.aspx <- for editing an account
/Users/User/Accounts/Account/Items/Default.aspx <- for viewing all items in an account
/Users/User/Accounts/Account/Items/Item/Edit.aspx <- for editing an item
Where can I read more about this kind of setup in a web application? Or, can someone point me in the direction of an available project that has this kind of layout?
View 1 Replies
Dec 21, 2010
I have 2 website A and B.
B site with windows authentication
I want to open B site as new window from A site, so its ask for windows credential. I have credential in A site. I am opening B site using Javascript.window.open.
how i can set credential for B site from A site.
View 2 Replies
Aug 8, 2010
I want to deploy a website in a computer where no webmatrix is installed. What are the requirement to run the website without webmatrix? And how do I configure IIS to run in startup with the site path?
View 4 Replies
Mar 28, 2011
i have an asp.net website which contains a link to a legacy site written in classic asp which has vb 6 classes (dll) in the back ground also.here is what i am trying to achieve:-i want to browse to the asp.net website using IE7,then when the link is clicked then enters the classic asp site i want to start debugging the classic asp and step through it.(also to make things worse, once a the vb6 class is instanciated, and a call make to its method i want to debug the method in the vb 6 code)My system :- XP Pro, IIS 5, Visual Studio 2005 (asp.net project ), Visual interdev 6 (classic asp project) Visual Basic 6 (vb 6 classes compiled into dll)i have been trying everything with no joy, there is lots on the net but nothing seems to have worked.i have the back office components installed so asp debugging is possible, i have configured the asp website s debugging is turned on.if i add a breakpoint to the classic asp site and hit the start button in visual interdev i can step through it fine.My problem is that if i attach interdev to a process (iexplorer.exe or dll.host) then the breakpoints are not hit. if anyone can help me with this it would be great.ont top of that, if anyone can explain to me how to also debug the vb6 dll code in visual basic 6 when it is called from the classic asp site that would be great!
View 1 Replies
Jul 28, 2010
I have 2 sites, one the main product site the other a secure site featuring the basket and checkout processes.
When a user clicks "add item to basket" I am creating a basket in a database and then adding the items to the basket. The theory was then to store the id from the database for the basket (the basketid) in a cookie, redirect the user to secure site, use the basket id from the cookie and display the contents in the basket.
However I am having trouble getting the secure site to use the same cookie. When I redirect the users to the secure site, the basket appears empty. I need to go back to the main site and then back to secure before the cookie appears.
Here is the code for the cookie
[Code]....
And then on the secure site this is how it gets the basket ID from the cookie
[Code]....
But like I say it's having problems. I've tried replacing the cookies with session variables as well but it didn't work.
View 1 Replies
Jun 15, 2010
I am working at a site not orginally devleoped by me. The problem is that all the pages are working fine and have no problem at my local site. but the site is not showing any content at live site from database. Niether it shows an error for it. How can I find the problem?
View 7 Replies
Mar 2, 2011
I have two web application. one is A and another is B. I want to send the data from Site B to Site A from javascript so that it should come as a pop up on site A. How can i achieve this thing? Is it possible or not? Need Response as soon as possible.
View 2 Replies
Oct 27, 2010
I made an aspx site with .net 4.0 framework through visual studio and it worked perfectly in the development environment but when I upload it to the server and try to run it I get this error: Server Error in '/' Application. Runtime Error Does anyone have an idea on how to fix this?
View 4 Replies
Oct 3, 2010
I am neither a server administrator nor very knowledgable about how to configure IIS, so I have the following problem:
My client hosts their public website from their own internal server running Server 2003 with IIS 6. The current website is a .NET 1.1 application configured as a website in IIS. Within this site a separate virtual directory configured as an application runs a separate .NET 1.1 app that serves as their online webstore.
I've been tasked with upgrading the website (not the webstore) to .NET 4 and getting it installed and working on their server. My initial plan was to simply change the existing websites home directory to the new home directory from the IIS admin console and be done, but then I read that this would break the application directory that hosts their online store because .NET 1.1 applications cannot be nested within .NET 4 applications. Can anyone confirm this?
View 3 Replies
Dec 2, 2010
I have an application that uses the ASPNET role provider.My SQL Server crashed so I took a backup of the database and restored it to a new SQL Server and changed the db connection string to that server.The web site runs but no users can login to the site.Is there something that I should have done when the db was on the other server?
View 6 Replies
Aug 6, 2010
I'm pretty sure the answer to this question is no, but I just wanted to get some feedback before I go down another path.
Here is my scenario. I have two websites. Website 1 is an internal website that cannot be accessed outside of our domain. Website 2 is an external website that can be accessed outside of the domain, but has access to webservices inside of the domain.
My question is, is there any possible way to display the internal page through the external page without making the internal page external.
View 1 Replies
Apr 16, 2010
I have created firstly ASP.NET MVC 2. and write more functionality. After I create asp.NET Dynamic Data Site. now, when I click on run button in Visual Studio, mvc app. opened in browser as [URL]. and asp.NET Dynamic Data Site as [URL]. but i want to merge this app. in one. can I use asp.NET Dynamic Data Site and asp.NET MVC-2 at the same time?
View 1 Replies
Sep 30, 2010
I hosted two asp.net web project in IIS with unique VirtualDirectory (sample and Sample2). Now i access a web page from Sample site and from this page i'm sending request to a web page in a site Sample2 with some values through Query String. I got the page and the values from query string.
Now my problem is, instead of sending values through QueryString is there is any other way to pass values from one site to another. Because, i never give the url for the second site to client. They know only the first site url based on user input i'll redirect it to the second site. Problem is user can see the values passed through QueryString and there is a chance to bookmark that link by user, it just like hacking. I donot want the user to do that.
View 2 Replies
Nov 16, 2010
We have a couple of ASP.NET applications running here at work and our users get the test site confused with the production site. What is the best way to help the users know they are at the test site. Is there a best practice for how this can be accomplished? Maybe have a custom control or something on the screen display "TEST SITE"?
View 1 Replies
Dec 14, 2010
<link href="../../Content/Site.css" rel="stylesheet" type="text/css" />
The above code shows an external css inclusion tag declared in Site.Master template.
My question: why did the Asp.net MVC team choose that way rather than using "~/Content/Site.css" ?
View 4 Replies
Jan 3, 2010
Is not a lot of fun of yielding much success at the moment.
When I copy my web site on the local machine some of the CSS classes in my themes are not applied. Although not major, I am concerned at what may be causing this.
My real problem is regard the root of the site. On my personal machin, I have set the root of my site using the tilde syntax. However, the root of the web site on my host's ftp site is wwwroot.
Does this mean that I need to change a setting somewhere for the page to be recognised?
View 4 Replies
Jan 6, 2011
When im try to export to excel from my web page im getting this error.
View 2 Replies
Mar 10, 2011
I am trying to publish a sugarcrm site I created in webmatrix to my site hosted on godaddy, however when i go into the publish settings and enter the server info, passwords etc., I can only get the connection to validate for the FTP protocol setting. When i click the button to test and validate the connection for webdeploy instead of ftp, it tries then says connection timed out. How do I get this to work so that I can publish my site on godaddy?
View 3 Replies
Jul 27, 2010
WebApp1: on IIS and configured with Windows authentication. Get User account from AD.
WebAPP2: a java web app on another windows box in same domain with authentication from AD
On web app1, I have a http handler like
public class MyHandler : IHttpHandler
{
public void ProcessRequest(HttpContext context)
{
string url = "http://WebApp2/Test";
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
CredentialCache myCache = new CredentialCache();
NetworkCredential netCredential = new NetworkCredential("myname", "mypassword", "");
myCache.Add(new Uri(url), "Basic", netCredential);
//...
myReq.Credentials = myCache;
//....
}
}
in above way, I can set authentication in code and pass it WebApp2.
But I don't want to put name and password in code. User info already available on WebApp1: in context.User I can find out the user info who already logined into WebApp1, so I want to pass this credential to WebApp2. I have tried to do following:
myReq.Credentials = CredentialCache.DefaultCredentials;
but I am failed becuast there is no data in CredentialCache.DefaultCredentials.
View 1 Replies
