Web Forms :: Website Search Efficiency For Large Data Sets?
Sep 28, 2010
The website I am working on has a section where users can search for content they have digitally subscribed to. way to cross-check both data sets, the first being content that matches the search criteria, the second being the content the user has subscribed to. What's making this is a mess is how the data is structured. Here is an example...
We store all of our content in a "Content" table in our database. A piece of content, a "journal", for example, and all of its child records (volumes, issues, and articles) are all stored as records in the "Content" table, each with a ParentId of its corresponding parent record. So, A journal can have n volumes. Each volume can have n issues. Each issue can have n articles. A user can have a subscription to any of these, which would implicitly give them access to all child records. For example, if a user was subscribed to a journal, they would have access to all of its volumes, issues, and articles. If the user was subscribed to an issue, they would have access to all of its articles, but not the parent volume or journal.
Some users own over 30,000 records, and we have over 100,000 content records in the database. The Content table has relationships with several other tables that are used for the search. This leaves me with an expensive query to find what the user has access to, and another expensive query to search through all of the records to find search criteria matches. Some of our searches take 20-30 seconds and I would like to speed it up to a max of 5sec per search.
I tried running a query to get the ContentIds of everything the user owned when they first visited the search page, and then caching it to eliminate further database hits, but when I passed the list of ints into the query via a linq .Contains statement, I hit the SQL parameter limit of 2100, since apparently .Contains() splits them all out.
View 3 Replies
Similar Messages:
Jan 7, 2011
I am working on a large project where I have to present efficient way for a user to enter data into a form.
Three of the fields of that form require a value from a subset of a common data source (SQL Table). I used JQuery and JQuery UI to build an autocomplete, which posts to a generic HttpHandler.
Internally the handler uses Linq-to-sql to grab the data required from that specific table. The table has about 10 different columns, and the linq expression uses the SqlMethods.Like() to match the single search term on each of those 10 fields.
The problem is that that table contains some 20K rows. The autocomplete works flawlessly, accept the sheer volume of data introduces deleays, in the vicinity of 6 seconds or so (when debugging on my local machine) before it shows up.
The JqueryUI autocomplete has 0 delay, queries on the 3 key, and the result of the post is made in a Facebook style multi-row selectable options. (I almost had to rewrite the autocomplete plugin...).
So the problem is data vs. speed. Any thoughts on how to speed this up? The only two thoughts I had were to cache the data (How/Where?); or use straight up sql data reader for data access?
View 1 Replies
Sep 13, 2010
It is a very large .txt file (more than 3M), and produced everyday, the content is user's system log like below:
2007-11-01 18:20:42,983 [4520] INFO GetXXX() SERVICE START
2007-11-01 18:21:42,983 [4520] WARING USER ACCESS DENIED
2007-11-01 18:22:42,983 [4520] ERROR INPUT PARAMETER IS NULL CAN NOT CONVERT TO INT32
2007-11-01 18:23:59,968 [4520] INFO USER LOGOUT
View 14 Replies
Feb 22, 2011
want to make fast search over 5,00,000 records in asp.net web application.
View 5 Replies
Mar 7, 2014
how to display large amount of data in griedview with serach funcationality.(the data should be 20 L)with example in asp.net.
View 1 Replies
Jan 18, 2010
I am trying to design a website in asp.net (just learning) and am having trouble positioning div's on the .aspx page.
I normally design in dreamweaver where you can move the div around freely and it sets the x and y coordinates for you but I am having trouble positioning an image.
Do you have to type the x and y coordinates in yourself in the code behind or in the css or is there an easier way where you can freely move the image around and it sets the coordinates.
View 5 Replies
Jan 13, 2010
I need to make a large Ebay-style list of listings (think of the page that comes up when you search for something in Ebay, with a photo, title, one-line description, price, that's paged, and sortable). My limited experience with DataGrids in the past always resulted in what felt like a sluggish page to me, but I'm not sure if it would have been faster if I had pieced it together with a data reader instead.So I'm wondering before I even get started, what is the fastest / most efficient way to retrieve and manage this data.Is it more efficient at runtime to use a DataGrid, or to write the paging and sorting in more manually and use a DataReader, or another way?Ebay manages to get thousands of listings to display very quickly, but they've obviously got a lot of manpower to squeeze every last bit of performance they can out of the site, and good high-powered servers.
View 5 Replies
Feb 22, 2011
I have been asked to build an ASP.Net website that has a custom search, when the page is loaded it displays a gridview with a list of data, then when a button is pressed it takes the value from the text box and runs a stored procedure using the value, the gridview then databinds successfully displaying the new data. But the select button does not take the rows data it takes the data from the row corresponding to the data when the page is loaded, not from the search button.
The code for the search is below.
[Code]....
View 3 Replies
Oct 5, 2010
I am trying to write a simple search form for our site. Here is my delima... I am trying to figure out that if there is nothing to search for in one textbox then search in the next textbox. The problem is that I dont know how to format my search string: For Example:
[code]....
View 5 Replies
Aug 7, 2012
I want to use website search in my website. which one will be the best?
View 1 Replies
Mar 6, 2014
How to play mp4 file in website which is having a size of approx 20 mb.
View 1 Replies
Jan 26, 2010
I created a website. I would like to include a search box that searches the whole website and returns the results.
View 2 Replies
May 7, 2015
I have one application and there are 1500 users of that application.When 200 users logged in application that time login page and other pages of applications are not opend. I want to manage 1000 of users at a time.
View 1 Replies
Nov 8, 2010
I am trying to implement a search solution for my website and was wondering whether there are other good search solutions I can use apart from adding a Google search box
View 1 Replies
Feb 26, 2010
just starting strongly typed data sets, I have managed to fill my data set and bind the gridview, but now I dont understand where to map the fields ? Inside the business logic layer or presentation layer.
working example to map fields with a select query and with an update .
my code for BLL file
[Code]....
View 7 Replies
Oct 7, 2010
I have a complicated report that nests six deep and so really hits the database hard. I want to use SqlDataReaders NOT DataAdapters and DataSets because I don't want big tables in memory.
MARS can get me the data fast, all from the one connection. But all the examples show it binding to a treeview.
I want to bind it to nested GridViews (six deep!).
Any examples of MARS binding to nested gridview/list/repeater controls of any kind?
View 1 Replies
Mar 22, 2011
I have two SQL queries that are stored procedures that have 2 variables that my web form will be supplying through text boxes. The two queries will produce two different results. I have read a couple of different of the MSDN articles online, including this one:
[URL]
that say that the tables must be similar to be able to merge them into a single datagridview. I know that datagridviews only like one table at a time, but I was curious if anyone had found a way to merge two dissimilar tables to one datagridview or not.
View 7 Replies
Aug 20, 2010
I have a C# website developed with Visual Studio 2008 with a SQL backend. I have a top Navigation menu that has seven different tabs. I have a specific right bar content for each of the seven tabs. The content for the right bar is stored in a sql database and will be accessed using data sets. I believe that I have two ways to display the information with the data set...
1. User control - Create a User Control for each data set and then drop that on to the respective pages.
2. Master Pages - Create a Master Page for each of my seven sections, adding the data set to the respective Master Page.
Is there a preffered approach to achieve my goal? I am thinking Master Pages would be the most efficient but not certain. Is there another method I should consider?
View 1 Replies
Oct 13, 2010
I have a following scenario - A use selects criterias from a webform (C#) - a result is then returned from the sql. Say, to start, there are 2000 data returned from the sql and are stored in a dataset.
Now, the user wants to choose a number of drops (up to 6) - with the number of drops selected, they can enter their own quantity in each box. So, they can divide the 2000 records into 6 different drops. With each drop, there is campaigncode created based on the mail date and the job number they provide.
So, now i have the 2000 records in a dataset. What i want to do is loop through the gridview where the user fill out the quantity depending on the number of drops (for ex: if they say 3 drops, then there will be 3 rows in gridview with 3 quantity box to fill).
In the loop, I am getting the quantity, dropdate and the job number. The Campaigncode is created as well. Then, i am looping through the datarow in the dataset; updating the Campaign Code column with the CampaignCode for the number of records thats set in a loop. It works but the Campaign Code is not populated for the next set of records.
Here is the code:
[Code]....
I think its this line right here thats causing the problem. But i don't know how to tweak this so as to accomplish what i want to do.
if (Convert.IsDBNull(DSAEFile.Tables[0].Rows[i]["Campaign Code"]))
View 3 Replies
May 10, 2010
I have a datalist and now i want to give custome paging to that datalist using multiple active result set. I mean i want to show paging as
first,previous,1 2 3. . . next, last.In this I want to get only 3 pages at a time. and once i got the page, I want to put it in caching and again if user click on that page number then i dont want to get the data from cache instead of hitting the database
View 2 Replies
Apr 7, 2010
Code:
<div id="searchbox">
<form method="get" action="http://www.google.com/search">
<input type="text" name="q" size="15" maxlength="255" value="" />
<input type="hidden" name="sitesearch" value="ieee-sb.uwaterloo.ca/" />
<input type="submit" value="Search" />
</form>
</div>
I'm using this code to create a search bar that will search only on the website above. It works when I put it into a html file, but when I put it in the rightnav user control in the ASP.net application, it only refreshes the page.
View 2 Replies
Feb 23, 2014
How to perform the search function for whole website in asp.net ....
View 1 Replies
Jun 30, 2010
i need acomponent to search in my web site easily like EasySearch or ultimate seatch
But all of them are trial
it is work in the localhost but not on the server
View 7 Replies
Oct 21, 2010
We have a fairly large website with several master pages. Right now the titles are being controlled by Session variables in individual pages and set by the master page. The code is bad and needs to be redone. With the current system, since the master page is executed last, we can't completely overwrite the page title when needed from the page. What is the best way to create breadcrumbed page titles across a large site in ASP.NET with several master pages, without using Sessions variables? Session variables are bad for page titles mmmkkayy.
View 1 Replies
Jun 7, 2010
i had a situation in which i want to implement a search within my website, to search information of my wbsite only and giving the list of pages which contain the information requested for seach.
View 5 Replies
