How To Large Product Catalog With Statistics - Alternatives To Sql Server
Apr 20, 2010
I am building UI for a large product catalog (millions of products).
I am using Sql Server, FreeText search and ASP.NET MVC.
Tables are normalized and indexed. Most queries take less then a second to return.
The issue is this. Let's say user does the search by keyword. On search results page I need to display/query for:
Display 20 matching products on first page(paged, sorted)
Total count of matching products for paging
List of stores only of all matching products
List of brands only of all matching products
List of colors only of all matching products
Each query takes about .5 to 1 seconds. Altogether it is like 5 seconds.
I would like to get the whole page to load under 1 second. There are several approaches:
Optimize queries even more. I already spent a lot of time on this one, so not sure it can be pushed further.Load products first, then load the rest of the information using AJAX. More like a workaround. Will need to revise UI.Re-organize data to be more Report friendly. Already aggregated a lot of fields.
I checked out several similar sites. For ex. [URL]. Not only they display the same information as I would like in under 1 second, but they also include statistics (number of results in each category).
The following is the search for keyword "white" [URL] How do sites like zappos, amazon make their results, filters and stats appear almost instantly?
View 3 Replies
Similar Messages:
Nov 21, 2010
I am wondering if there are any good articles you could point me to that would present the implementation details or options for delivering advanced search capabilities. Like what we see at [url] and similar Web sites offering vast numbers of products to be searched. The search process for these starts with a simple "Search" text box - usually with auto-suggest. Then the user can drill down and narrow the search by [for example] price, manufacture, size, capacity, and other relevant attributes of the product being searched.
When I navigate such sites and search for products, drilling down into the various categories, it's easy to think that there is some complex engine behind the scenes that generates dynamic SQL to satisfy the various and potentially extensive criterion required to satisfy the search requests.
Or could it be simpler than that? In any case, I understand this question may not have a specific answer; so I'm hoping for some article or reference implementation I could study. Short of that, if any of you have implemented advanced search of a product catalog (presumably searching products stored in a reasonably normalized database; through 3NF or so); any critical implementation considerations would be appreciated. I'm particularly interested in the meta data that may have to be present in the underlying data set in order to satisfy some of the search requests.
View 2 Replies
Jul 6, 2010
i m developing a product Catalog. All fine, i want to manage wish list for product, which user wish to see, now my question is that how can i manage this wish list, coz i have no uniqueness of the user, coz ny one whether it is registerd or not can wish the product in my scope, i thought to distinguish the user by their IP address becoz this wish list is temporaray, how can i delete the record when user close the browser if go to the IP address for distinguish..
View 2 Replies
Jun 29, 2010
Is there any good code (paid or open source) out there for:Handling PPC ads on our site? This would (hopefully) include pages to create ads, reports of ads run, determining what ad to show, and tracking billing.Adding comments to our site? This would (hopefully) include up/down/inappropiate votes on other's comments.A product catalog? This would (hopefully) include advanced search and entering product items.
View 1 Replies
Feb 28, 2011
I'm looking for a way to add 3 level product catalog to cart based on MVC musicstore sample:
root product categories
sub categories
products
Clicking in catalog item should show subitems with pictures.
Control is placed to Site.Master. It can be treeview in cart left side or horizontal menu in upper part of screen.
Data is read from database from controller.
How to implement this ? Which free control is best for this or is there sample code for this ?
View 23 Replies
Mar 2, 2011
I am trying to create statitics for game downloads and am having dificulty with a sub part of my query. the goup by command is not having the disired result and i think i have been looking at it too long to see what i am doing wrong. [Code]....
At the moment it is producing each result in many rows when only 4 rows should appear. its the download column that is doing it. needs to be sub queried or something to display total in relation to month,gameid,userid.
Month-download-GameID-UserID
1 1
1 1
1 1
1 1
View 2 Replies
Dec 10, 2010
title isn't as detailed as I hoped but I have a few questions. Traditionally I've always been a mixed mode person, storing PDFs and other things into the file system with pointers from my SQL DB to them.
Recently I've heard more people storing files into the DB directly and wonder if I'm storing files around 100-200MB in size, will I run into a lot of issues? Are there ways to compress the files other than perhaps zipping them up and storing the zip?
Finally, something I've always wondered about but never really found a good work around for... if I decided to continue with the mixed-mode storage solution, and these I need to retrieve these files for use on a website, how do I secure these files so that someone can't just randomly come and retrieve these files off the server by guessing files names, etc?
I did one work around once where I stored a set of PDFs to a non-web directory like c:pdf and setting permissions for the web account to access them. This allowed me to retrieve them using the references in the DB and provide some garbled filename so people couldn't retrieve it themselves but this all seems rather complicated.
View 1 Replies
Jan 24, 2011
I have a multiple database operation, Have three tables (MSsql) and need to delete one row in each table in one operation, one after another. (Three functions, three rows of code). Everything is working as it should be, recently I was advised to use a transaction scope, which should guarantee the safety of deletion from all 3 tables together.
Here is my new code:
[Code]....
Locally all worked just fine, the transaction scope worked well, but when I have uploaded my application to my hosting server, I got an error of: The partner transaction manager has disabled its support for remote/network transactions. (Exception from HRESULT: 0x8004D025).I talked with the host support, they tried to do something, then I got this error: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)).
After a 2 days, They have contacted me, they said it is not possible to disable some restriction, that will allow an access to this operation, because I'm using shared hosting, and I need to upgrade my plan to VPS. VPS is a very expensive for me, and I can't afford it, What I'm looking is a some similar way of doing the same operation as transaction scope do, that may work at my hosting, At least I would like to try something else. Is there any additional way for the transaction scope?
View 1 Replies
Jun 22, 2010
I'm looking at options for rendering PDFs on the server to avoid the user having to download them (they're big docs) and/or have a PDF reader installed and configured (audience is not tech savvy). The Google Viewer and Scribd are nice examples, but I'm not interested in making the PDFs available on the web (for Google) or storing them on another party's servers (Scribd).
My favorites right now are FlexPaper and PDFWebViewer.NET but I wanted to see if I'm missing any other options. Most of the related SO questions on this topic are somewhat dated, so maybe there's something new?
Any other options to check out?
View 7 Replies
May 5, 2010
I've got an ASP.NET 3.5 Web Forms application in which a large chunk of code needs to be duplicated between a few different pages. Sounds like the ideal candidate for a user-control right? Problem is, this cannot be contained within a <form runat="server"> because it contains a client-side form of it's own. There are no runat=server controls or postbacks or anything that really need that webform - think of it just as a chunk of HTML with a few basic <% %> tags. I'd just want to set a property on the control when it's loaded, so that it knows what to output. This is purely an exercise to make the code easier to maintain.Before I resort to using an oldskool <!--#include-->, is there some better way of doing this?
View 5 Replies
May 11, 2010
Is there any web service which will return the prouct name by accepting the product barcode input?
View 2 Replies
Oct 7, 2010
On our web server we have a search form that fires off multiple searches and combines the results into 1 page. The main search is of the indexing catalog for our web site. We have used this for several years and it works just fine. The company just sponsored a weight loss program and we have discovered that the word "weight" returns an error. The word "weigh" works fine but if we add that "t" on the end it fails. Here is the error:
Incorrect syntax near 'weight'. Expected '!', '(', NOT, ISABOUT, FORMSOF, STRING, PREFIX_STRING. SQLSTATE=42000
This is the query at the time of execution:Select DocTitle,URL from SCOPE() where (CONTAINS(CONTENTS, 'weight') OR CONTAINS(DocTITLE, 'weight')) order by Rank DescDoes anyone have an idea of how to get past this problem? I'm guessing that "weight" is probably a key word or something similar.
View 1 Replies
Jan 29, 2011
I am trying to convert a asp page to asp.net.
I changed everything except this one line Set dbC = Server.CreateObject("ADOX.Catalog")
Can anyone suggest an equivalent to this ADOX.Catalog in asp.net This dbC is being used in the following way in asp
[code]....
View 2 Replies
May 7, 2015
i am using asp.net webform not asp.net MVC.
i want to make my webforms as single page application
ex: i have a webform for products and the correspoding Product Links
if i click on any products the related details of that product has to be dispaly in the same webform .
i dont want to goto another webfom and dispaly the details of the product.
View 1 Replies
Jun 9, 2010
I have a product category table with the following fields:
cat_id (PK)
Cat_name
Cat_desc
Parent_Cat_Id
Now when a user wants to add a product he should be able to select multiple categories from a listbox(multiselection enabaled). But to let the user know the hierarchy of the categories, I need to display them in the following style:
parent category 1
parent category 1->sub category 1
parent category 1->sub category 1->sub-sub category 1
parent category 1->sub category 1->sub-sub category 2
parent category 1->sub category 2
Parent category 2.........
I know that to achieve this I need to use recursive programming. But how can we do it via stored procedures?
View 2 Replies
Aug 8, 2010
I have a sql table (ProductsOffers) in which I store all the offers posted for each Product it consists of columns
offerId int
ProductId int
ProductSpec nvarchar 50
OfferDate DateTime
OfferId + ProductId = My primary Key
everyday there will be many offers for each model and some of them may have no offer in this day. Now Suuppose that I have 10 Products with 15 offers. I want a sql sentence or function that let me select the latest offer for each product. When I select by date it returns the last inserted offer and they may have repeated products and also there will be some products not appear in the result. how can I get a result with the latest 10 Offers for all my products with one offer per product
View 8 Replies
Feb 6, 2011
I want to generate 30,000 cards and each card must be duplicate check with database. In my card, there are 2 things. Serial No and CardID. If any card already exists then I generate another card id but with the same serial no.
So how faster way I can generate 30,000 card with duplicate check? Which one I have made application, it takes about 25 minutes to insert.
View 33 Replies
Jan 3, 2011
In my program, I need update a filed to the database. The field is for a pdf file content. I know, maybe I should save the pdf file to the local file system, and save the file path to the SQL. but, right now, it is the file content saved to the SQL.
from my program, it loads the file which is input by user. I use File.ReadAllBytes. Then, run the stored procedure on the SQL server, and use the bytes[] as parameter, to insert the file to the SQL. it works fine when file size is small. However, as file size becomes larger, say 200MB, sometimes, it pops out of memory exception.
So, I'm thinking, maybe I can load parts of the file, say 1MB a time, then update the database, then loop. so, this way, no matter how large the file is, it should not have any problem.
my question is: is there a standard way to do this? so, I should not save file content to the SQL at all? shoudl save file in file system, save the file path to sql only? or there are other way to deal with this?
View 1 Replies
Feb 24, 2011
We have a web service method which accepts some data and puts it in Lucene index. We use it to index new and updated entries from our asp.net web app.
These entries are stored in a large SQL Server table (20M rows and growing), and I need a way to be able to reindex the whole table in case if current index gets deleted or corrupted. I'm not sure what's the optimal way to retrieve chunks of data from a large table. Currently, we use the fact that the table has PK which is autoincrement, so we get chunks of 1000 rows until it starts to return nothing. Kind of like (in pseudo language):
i = 0
while (true)
{
SELECT col1, col2, col3 FROM mytable WHERE pk between i and i + 1000
.... if result is empty 20 times in a row, break ....
.... otherwise send result to web service to reindex ....
i = i + 1000
}
This way, we don't need to SELECT COUNT(*) which would be a big performance killer, and we just move up the pk values until we stop getting any results. This has it's con: if we have a hole greater than 20,000 values somewhere in the table, it will stop indexing assuming it reached the end, but that's a tradeoff we have to live for now.
way of getting data from a table to index? I would assume we are not the first ones facing this problem - search engines are widely used nowadays :)
View 3 Replies
Dec 18, 2010
i want to to sotre large amount of character in one field of a table i use text-nvarchar(max) and but it show me this exception
(String or binary data would be truncated The statement has been terminated.) I know that text and nvarchar can store 8000 character
View 7 Replies
Feb 2, 2010
I have a catalog that hase standard pricing now i need to increase this to 5% for each product. this is the way i bind the data:
Price:<%#Eval("Price",
"{0:c}")%>
View 3 Replies
Feb 9, 2010
I have a requirement to upload large files using dotnet using ftp. Any opensource programs available?
View 5 Replies
Feb 17, 2011
Handling large number of images downloaded from server
View 4 Replies
Dec 27, 2010
In my database when I fire query it takes 40 secs on 1 crore data, similar when I use join with other table then it take more time. I have taken care non cluster index such thing. But still I want to optimize my query, what other thing I need to take like buffer, disk size etc. I am not sure on this area.
View 11 Replies
Feb 17, 2011
I have a DTS script which transforms a Tab delimited text file to a table. I get the following error when trying to transform the data of a field that is greater than 255 chars:
[Code]....
I have seen this issue with importing from Excel with the jet 4.0 engine, however I am importing a text file.
View 3 Replies
