Allow Search Engine To Crawl Usernames?

Apr 28, 2010

I have a site where users can enter their profile and password-protect certain details. I would like search engines to crawl the 'unprotected' parts of the profile (which varies from user to user). Similar to how if you enter a user's name in facebook, their Facebook profile comes up in the search results. Do I have to do anything special to ensure that the bot doesn't crawl the password-protected sections, but still crawls the (always-public) username?I'm not sure if this is even an issue, but I'd like to update my robots.txt to allow for this.

Also, how do I ensure that the usernames are available to the bots (in a safe manner)? Do I have to create a separate directory with a list of names, or is there a better way?

View 2 Replies


Similar Messages:

SQL Server :: Minimum Search Engine - Add A Search Mechanism To The Site

Jan 19, 2011

I have a database filled with information about a bunch of different articles. I'm trying to add a search mechanisim to the site where you would type in what you are looking for, and it would search through the articles finding the most appropriate article. In my database I have a bunch of different fields that will help find the best article. For example, for each article, there is a title, description, category, a bunch of tags, Number of page views, number of votes, and the sum of all the vote scores. All this informaiton i believe can be used to gather the most relevent article based on the persons search terms. The problem is I don't know what to do with this information. What I would like to do is have a scoring system where the highest score is the first result and the lowest score is the last result. Where the first result is the most relevent.

So for example: The user types in "Korean War" it looks through all the articles, and gives all the articles with "Korean War" in the title + 1, all the tags with "Korean" or "War" + .1, then adds the number of views/1000, and then adds the average vote/4. (I just made that scoring system up on the spot, I would need to do some testing of course). Then it would sort the articles based upon the score it recieved. Is there anyway to do this with a select statement where I can run all of the calculations and then sort them with an "ORDER BY" command? Or would i have to have the server side code do the calculations store them in an array, and then sort them in the array? Or is there a better way of doing this that I might not know of (which is certainly possible).

View 3 Replies

Build A Simple Search Engine Which Search From Specific Websites?

Dec 5, 2010

I want to know how can I build a simple search engine which search from specific websites, for example it should search codeplex and msdn only and show the result on my page?

View 1 Replies

SQL Server :: How To Save Pdf Files In Database And Create A Search Engine To Search For Pdf Files

Sep 28, 2010

in my project i would like to save pdf files in database, how to save pdf files in database ? along with that i would like to create create a search page to search for pdf files with PDF file names , how to accoplish these things

View 3 Replies

Web Forms :: How To Add Search Engine

Aug 25, 2010

I had gone thru many discussions re: search engine & didn't get the answer to what i intend to design. I intend to put the search textbox & button to the masterpage so that whatever page the user is within the website, he/she can do the search.

I've been following the example (SearchDotNet) in the developerfusion but I just can't figure how I'll make it to redirect to the searchsite from the page the user triggers the search. Do I put the codes in the masterpage?

View 3 Replies

How To Implement Search Engine

Apr 8, 2012

How to implement search engine Within a Partucular website that I supposed to build using asp .net C#.......

View 1 Replies

Trying To Make A Search Engine For Website

Apr 25, 2010

i am newer to asp.net and i am trying to make a search engine for my site like this one [URL]. i am able to get the SEO friendly URL like this with the keyword and redirected to the Search.aspx.

Now how can i get result like this, i need some assistance to start. i have used contain attribute in LINQ query and get results. but i want to split the search term and i don't know how and then want to search the each word using contain attribute and then want to get results like this guy. you can reffer me anything tutorial or any componenet which is made for asp.net c#.

View 1 Replies

How To Get Google Search Engine Webservices API

Aug 16, 2010

i need the procedure of how to get the GSE API from google and use this API to my code

View 1 Replies

URL Rewriting For Better Search Engine Optimization?

Jul 21, 2010

I am working on ASP.NET website. I need to post a Question Paper based on following parameters.

University
Branch
Subject

At Present My URL is in the below mentioned format

//localhost/MYASP/Posted?PostId=**

My URL should be in the below mention format after URL Rewriting

//localhost/MYASP/Posted/University/Branch/Subject/TITLEOFThePost

let me know the process of finding my querystring after url rewriting

View 2 Replies

Web Forms :: How To Create A Search Engine

Dec 22, 2010

I would love to create a search field on my master page that returns a page full of results.

It seems simple to me. I need it to search each of my tabels. aka(Customers, Loan Information, Real Estate, Vehicles) I want a repeater or datalist of sometype that hyperlinks to a details view page for each of the tables(easy to do).

I don't know the best way to tackle this. It is a local intranet application so I was thinking about possibly just passing the words or numbers in the textbox to a query string then running a sql command for each of the tables and returning data in a datalist for each table?

Would that work or is there a simple way to do this? It would be nice for it to highlight the match? What should my sql commands look like?

View 3 Replies

C# - How To Use Bing As The Search Engine On Site

Jul 15, 2010

Does Bing has an option similar to Google Custom Search or Yahoo where I can use Bing to power the search results on my site?

Couple requirements:

Works with an ASP.NET site (is a .NET project) Host the search box and results on my own website Ability to customize the look and feel of the results to match my site (Full control is ideal but I understand it's not possible with free solutions)

I did a search for Bing custom search and found this: [URL] but it's not exactly what I am looking for.

View 2 Replies

Filter Out Search Engine Bot Traffic?

Jun 16, 2010

I'm doing custom-rolled view tracking on my website, and I just realize that I totally forgot about search bots hitting the pages. How do I filter out that traffic from my view tracking?

View 2 Replies

Architecture :: Search Engine For Website?

Oct 11, 2010

I need to create a search engine for my website that searches my entire content and files stored in a database.What is the best way and easy to accomplish this?If can, please provide a few samples or links.

View 4 Replies

Architecture :: Search Engine Implementation?

Jul 22, 2010

not sure if I'm posting to the right thread or not.. but here is the questiondoes anyone know how to implement a best pratice on SEO?One of the example is: if the user types "brand new canon camera" then I want the system to find a sequence of string in the database that contains "new", "cannon", "camera". It is ok with or without the word "brand". However if the user types "HP laptop DV1000" and i want the system to find all HP laptop whether new or not but limited to specific series.

View 3 Replies

Search Engine Support Keywords?

Jun 14, 2010

this is naresh.i have some query.in google search engine while typeing any letter based on letter some support keyword is getting.

View 8 Replies

Web Forms :: Search Engine On Every Page

Aug 10, 2013

how to embed a search engine into all asp.net page? i want to add a search engine which allow user to input key words then display some title which has the keywords. 

View 1 Replies

Finding Website Search Engine Function

Jan 28, 2010

what I'm looking to do is to get a web site search function where I can enter the URLs and body text myself rather than running a spider unless the spider can handle my url querystring structure. Obviously, something automatic would be best but I'm not too choosy as long as it is free.

View 2 Replies

Search Engine - Deny Google Indexing?

Mar 23, 2011

Is there a way to prevent search engines from indexing a site by modifying the ASP.NET Web.config file?

View 2 Replies

Updating Google According Search Engine Optimization?

Feb 26, 2011

i have awebsite and i made all my steps to make it in the first links on the search of google (SEO) and now i want to go to google updates to make the steps that apply the website on google after one day almost. can any one tell me what i should begin with or do?

View 7 Replies

Terminating Index Process For Search Engine?

May 22, 2010

i am working on a small search-engine app with asp.net 4.0. the indexing process can index each URL founded in each document and this recursively performed for each other. now how i can terminate this process when i want or when user clicked on stop browser button?

View 1 Replies

Web Forms :: Website Not Searchable By Search Engine

May 8, 2010

i m making website in asp.net and my website is not searchable buy search engine i use this in master page

<meta name="description" content="Trump Academy offers various Courses in Aviation,Ground Staff,Travel and Tourism & Hospitality"/>

View 3 Replies

Web Forms :: How To Make Internel Search Engine

Mar 19, 2010

i want to make "Internal Site Search Engine" for my website

View 1 Replies

REST Interface Slow Down Search Engine?

Jan 25, 2010

To get a search website built quickly I plan to split the work between two teams: One to build the search engine and one to build web UIs (mobile/desktop). My plan is to build the search engine as a set of REST services based on .NET 3.5. UIs may be built using some other technology.Questions: is the REST interface likely to be a performance bottleneck? How best to avoid this?

View 3 Replies

Architecture :: Create An Image Search Engine?

Sep 14, 2010

how will I able to create an Image Search Engine?

It is about using an Image and search similar images on web.

View 1 Replies

C# - Options For A Search Engine Database On Windows?

Oct 14, 2010

I have a project to create a high traffic search engine similar to altavista.com. The windows .NET C# will be used for the project. I am looking for a good search engine database that can handle a very high load. I have taken a look at lucene and sql server 2008. I have read that lucene tends to get corrupt when the load is very high. So, I am considering sql server 2008 but I am uncertain sql server can handle a very high load over gigabytes of data.

The database will be given a chunk of data and needs to fetch similar data elsewhere in the table. it will act just the similar questions area on stackoverflow. It will need to search over millions of rows. Is sql server 2008 and lucene my only options

View 1 Replies







Copyrights 2005-15 www.BigResource.com, All rights reserved