Google Bots Looking At Some Pages Via The Robots.txt File?
Feb 11, 2011
I have learned recently that I can stop Google bots looking at some pages via the robots.txt file.Great got that sorted out.However how can I stop the crawler or bot from scanning some text on my page. But reading other parts just fine.For example I have a large legal disclaimer on each page,and I have seen that via google web master tools that my keywords are all out of the disclaimer.Cant have that!
Can I turn off a control or label to crawlers eyes?? Any ideas..I cant move disclaimer to a single page as then it looses effect.
UDPATE: <!--googleoff: all-->Come to the fair!<!--googleon: all-->
we have a situation where we log visits and visitors on page hits and bots are clogging up our database. We can't use captcha or other techniques like that because this is before we even ask for human input, basically we are logging page hits and we would like to only log page hits by humans.
Is there a list of known bot IP out there? Does checking known bot user-agents work?
I have a website which has two domains added. Both domains point to the root of the website. Is it possible to alter the robots.txt so that one of the domains doesn't get crawled, while the other still does?
I got a bot and screen scrapper problem. I have successfully redirected them somewhere else, but they still show up in my stats. I run my redirection code in the pre init section of the master page. I use discountasp for my hosting provider. Is there a way to run my code before the preinit page section?
I have an online application that all of its pages use HTTPS. I have 3 questions:
Does Google index HTTPS pages?I have a password protected single ASP.NET page (using HTTPS). Password protection is basically achieved by a Session object. When the correct password is entered, it hides the login panel and displays the same page which has a sensitive data. Is this page indexed by Google? I have a Secure folder that I implemented Forms Authentication. All pages in folder use HTTPS as well. Are the pages in this folder indexed by Google?
This is how I added an email link on my page, but does .net have any built in tool for preventing Robots from finding this address? What is a good way to present an email link that wont be detected by spam robots?
I am trying to do the following I have a dynamic table in my asp page and I want to show the table in multi pages like google results note:I don't to use gridview so any another way ??
I have used Google Plus one in my web pages in my website. I have used this in a User Control Page and called this .ascx page into my all .aspx page. I have used the code like below:
<g:plusone size="medium" width="120"></g:plusone> <script type="text/javascript"> (function () { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })(); </script>
It is working fine in my pages except 2 pages in which it is not showing. When i see the view source of the page then i find the above javascript and HTML code. I have called 6 methods on the page load of these 2 pages. what is the reason of Google Plus one not showing on my this 2 pages. what i will have to do to show Google Plus one on my this 2 pages. I have called this User Control page on my other pages.
Validation summary is giving me problem when using Google Chrome. On all pages of my Website it truncates the error message at the end, For example it shows:
I'm wondering what would be the best solution for hidding a folder like for example "AdminFolder", and also .aspx pages inside this folder.
I have several pages inside "AdminFolder" which I (as "Administrator" :) ) plan to use for some background work on daily basis! I am using Membership shema and no one except me can not / or should not acces this area, but I'am afraid if web crawlers like google find and expose this part of my page in searching results!
I also need to create a second folder for PDF files which I also would like to hide (incl. PDF Files inside it) from web crawlers!
I'm trying to add Google Adword conversion code script to certain aspx pages in our website but I'm running into an issue with the site using master pages. The Google instruction said to place the code before the body tag but with master pages being used the code will be on all the pages using the master page. I would like setup it up where certain pages use individual conversion codes with others not using anything. Any suggestions or examples would be appreciated. Also, I'm using C#.
i have used file upload on one of my web page..the file path is not shown in google chrome..rest in all browser the path is shown..in google chrome only the file name is shown and not the whole path..i want the path to be shown in google chrome
I'm rendering charts on my website using the Google Charts API and I want to export these into a report for the users, most likely a PDF file. What methods are available to do this programmatically into a PDF file?
I want create/show/edit a excel file from my asp.net application. All I want is like the working of Google docs. Is there a way I can implement it? Any dll or can i even use the google docs api for that?
In chrome it open new tab but shows popup block url and ask for always allow or block url then we have to give manually always allow and then click on link again open url but in other browser it not even open new tab
I want that whatever link i open in new tab it should be allowed from the code itself. No manual setting should be required.
As an organisation, we use Google Apps. We have the paid version (mapped to our domain) etc...We are developing a web based application to manage orders, and other business functionality.I want to be able to use federated login with our google apps accounts- For example, if a user is logged in to their email (gMail) - they should automatically be logged in to our ASP.net application
If they're not logged in - the log in form should auth. against our google apps account.How can this be done?Is it possible to be able to "get" the user who is currently logged in using this method etc...?
I want to know how to upload local file to google drive without using file upload or html input file control.. i have file path and other information. I am using web application and vb language...
I've implemented a google map with points and stuff that uses an address that the user inputs. When you click a point, the popup bubble appears with the name and address in.Often this name and address is a prominent location, as it's used for meetings and things, such as a university. If you google the address yourself on maps.google.co.uk then you get google's own popup bubble, which often has a photo, information, opening hours, links to directions, reviews, etc. etc.I am wondering if there's a way to use that popup dialog instead of my own, where it is available. I can't see anything in the API to do this.I'm using V2 as we support IE6 in a lot of our users, but I've been told recently I can upgrade to V3 should I need functionality from it.
just want to add the effect of map momentum by the dragging of map as in maps.google.com has. currently on my map where i stop dragging the map stops there but in maps.google.com if you drag a mouse the map will not stop there it will continue the movement in the same direction for a second or few cordinates (i dont know basically), but I want the same thing on my google map. I am using GMap2.
I just want to ask something about master page. My requirement is that I want to use two master pages like different banners and want to share for same content page for easy maintenance purpose. Master Pages will be changed depend on login users.
how can i specify two different login pages in root web.config file since i need to have authentication for two folders.for securing My Account module i did like this in the root folder i need to have it for another folder called EBox also.
I want to download a file from google drive (like .docx, .xlsx etc.) to a specific location, I am able to do so which are publicly share using WebClient object.
But now I want to download file which private in share but unable to do it.
How to improve my code which should download private file also.
//Parameters //1) Link - google drive link from where to download //2) filename- name of filename- //3) oringinalDocDirectory- path where file has to save
public ActionResult SaveToServer(string Link,string filname,string oringinalDocDirectory) { if (!Directory.Exists(oringinalDocDirectory)) Directory.CreateDirectory(oringinalDocDirectory); var docFinalSavePath = Path.Combine(oringinalDocDirectory, filname); WebClient client = new WebClient(); client.DownloadFile(new Uri(Link.Replace("&export=download","")), docFinalSavePath); return Json(new { status = docFinalSavePath }, "text/html", JsonRequestBehavior.AllowGet); }
