Practices For SQL Inserts On Large Scale In Reference To Ad Impressions?
Feb 20, 2010
I am working on a site where I will need to be able to track ad impressions. My environment is ASP.Net with IIS using a SQL Server DMBS and potentially Memcached so that there are not as many trips to the database. I must also think about scalability as I am hoping that this application becoming a global phenom (keeping my fingers crossed and working my ass off)! So here is the situation:
My Customers will pay X amount for Y Ad impressions These ad impressions (right now, only text ads) will then be shown on a specific page.The page is served from Memcached, lessening the trips to the DB When the ad is shown, there needs to be a "+1" tick added to the impression count for the database
So the dilemma is this: I need to be able to add that "+1" tick mark to each ad impression counter BUT I cannot run that SQL statement every time that ad is loaded. I need to somehow store that "+1" impression count in the session (or elsewhere) and then run a batch every X minutes, hours, or day.
Please keep in mind that scalability is a huge factor here.
Here's the scenario: Buyers and Sellers (where all of the sellers have a unique D&B number).
I'm catering to the buyers, and I want them to be able to import a list of sellers based on the D&B number.
Here are the tables:
BuyersSellers - unique list of sellers - one record, no matter how many buyers work with them.BuyerSellers - lookup table - buyer_id, seller_id sets which buyers work with which sellers ImportedSellers - temp table that holds sellers the buyer wants to add.AllSellers - master table that has all known sellers with D&B numbers (remote database).
Here's the process:
Buyer uploads list of new sellers, they get added to ImportSellers tableMatch ImportedSellers against AllSellers (valid), ignore unmatched (invalid).Add valid ImportedSellers to the Sellers table if they don't exist.Create BuyerSeller records for all valid ImportedSellers - using data from AllSellers.Delete all valid ImportedSellers, leave the unmatched behind so they know what they need to fix.
I've got the upload and import working, but now I'm a little stuck on how to do all of the matching and inserting, and do it as quickly as possible.
I'm thinking of using ASP.NET and MySQL together on a large project. Does anyone have any experience in using these two on a large scale and are there any aspects I should be wary of? If there is a chance that there could be compatibility issues, then I may be prepared to purchase an MSSQL license, but I'd rather go the free route right now. In theory it's possible, but how easy is it to transfer a MySQL database to and MSSQL database in the future?(I'll make this a CW if requested).
For WPF, there's the Microsoft Patterns & Practices's Prism project.
Prism provides guidance designed to you more easily design and build rich, flexible, and easy-to-maintain Windows Presentation Foundation (WPF) desktop applications, Silverlight Rich Internet Applications (RIAs), and Windows Phone 7 applications I was wondering whether a similar project (reference implementation) intended for software developers building WCF applications exists.
In our main internal project (a .Net WinForms rich client app), we don't talk directly with the database but instead fetch and update data with ASP.Net web-services that we also control. Our current setup is giving us some bottlenecks. For a new smallish project, we want to try WCF. Objective question: Where do I find a not-too-basic WCF reference project?
based on a recent article on MySpace blaming the microsoft stack not being able to easily scale here.Is this true? Is scaling an app built on the microsoft stack (asp.net mvc here) nearly impossible? does php/LAMP really scale better than microsoft?
I maintain a web application (ASP.NET/IIS7/SQL2K8/Win2K8) that needs to access documents, actually hundreds of thousands of documents, and growing. Currently, they are all on a Windows 2K8 Server fileshare, being accessed by UNC path (SMB). The files are in a single flat directory and I'm trying to plan how to best improve this solution. I don't want to use the SQL Filestream attribute as it would be significant effort to migrate it all into that, and would really lock in to SQL Server. I also need to find a way to replicate the data for disaster recovery, so perhaps a solution can help with that too. Options could be: Segment files into multiple directories? application would add metadata for which directory it's on (or segment by other means) Segment files into separate servers? (virtualize)Backup becomes more complicated. Application would add metadata for which server it's on
NAS Storage SAN Storage
Put a service (WCF) in front of the files and have the app talk to the service bonus of being reusable across many applications Assuming I'm going to store on filesystem and not in database (I've read those disccusions here), which would be a more scalable solution?
I am using abcPdf to convert an HTML report into a pdf file. The pdf has to be a single landscape A4 page.Do you know if there is any way to tell abcPdf to scale the HTML page to fit on a single page in the pdf? I tried using the Magnify() method, and it scales the content but still breaks it into pages, even though it would fit on one page. I've been scratching my head on this for a while now
i wants to upload images on server with small scale, large scale and medium scale on server hard disk when user uploads picture on server by upload image click..
There is a database (SQL Server) with locations (longitude and latitude values). I need to plot the route on the map (Google or otherwise) based on those locations extracted from database. Plotting lines and map are on ASP.NET webpage.
What would be the most and cost-effective way to accomplish the task above?
On my X Axis, I have months. The chart shows up to 11 points, i.e. Jan - Nov of the same year, but when I add 12 points (Jan - Dec), it will do an auto label thing and change the interval for every 4 months.
How can I change the graph so that it shows 12 months before it does the auto labels?
Here is the server control code I am currently using.
I have a formview and I opened it in Insert mode where the fields were all blank as I expected. I then entered some data into the fields but instead of clicking insert I clicked cancel. I then entered the formView in Insert mode again and the previous data was still present.
how to set it up so it will be cleared if cancel was pressed?
I have a group of radiobuttons. The field they are 'bound' to is of datatype tinyint (sql server), byte .NET. It is set to NOT accept nulls in the database column (col1) and no default value has been set for the column, or indeed, anywhere else.
If the user doesn't select any of them, I would expect null to be that the model would attempt to insert into the database. I would therefore get a validation error that I can display to the user, prompting them to choose one of the options.
However, instead of null, the model inserts a 0 into the database. As this does not fail, no error is raised.
I am not setting the default value to 0 anywhere, not in the mvc app or the database
First: Using the wizards in Web Developer 2008, is it possible to create multiple inserts into various tables? Just looking for a Yes/No answer.What is the easiest, and best way to create a SQL query to search multiple tables, using a dropdown list as the criteria? Using 3 tables as an example: How would this look? Anyone know of some examples out there with regards to querying or updating multiple tables?
I'm having a question about the LinqDataSource. Does anyone know if it's possible to perform multiple inserts with the LinqDataSource? Let's say for example that I have 2 tables.
Table A AID = PK autoincrement Name AddressA FK AddressB FK Table B BID = PK autoincrementStreetNumberPostalCodeCountry
These tables are just examples to clarify my question. Let's say that I have a FormView with a LinqDataSource where the user can enter his name en a billing address (A) en delivery address (B). Now I would like to know if the LinqDataSource is able to insert 2 records in table B and use these ID's to insert them in table A together with the name?
I am trying to insert a new account in my Acccounts table with linq. I tried using the EntityModel and Linq2Sql. I get no insert into my database nor an exception of any kind.
public static Linq2SQLDataContext dataContext { get { return new Linq2SQLDataContext(); } }
I'd like to use sql to select top (#) random row into another table, but it inserts the same record a few times sometimes. I want to avoid duplicates to be inserted, what shall I do?
what I've got for now is
INSERT INTO CompWin (PrizeID, EntryID, UserID) SELECT TOP (50) PrizeID, EntryID, UserID FROM CompEntry ORDER BY NEWID()
I'm trying programatically to create a new user, that's working OK on it own, but whenever I add the following line to my code, it seems to create two new members with the same name in my aspnet_Users table?
I have been struggling for a while 72 hours with this .When I update the form view a NULL value is inserted in my product ID field instead of the productID from the drop down list.
I have a masked edit C99999 as a mask and allow a "C" or "D" as the beginning character.This C or D is added to an existing number when that application is Denied or Canceled.The trouble is the masked edit is in overwrite mode and destroys the original app#.Can I make it INSERT or do I need to do an append of the original number to the newprefix in the update code?I hope this isn't another RTFM question... <BLUSH>
I've updated a number of production sites from 3.5 to 4.0 and have found a really annoying problem where it will insert a blank space into Multi-Line text boxes
The tag is entered as.... <asp:TextBox ID="AdvDayNotesTB" runat="server" TextMode="MultiLine" CssClass="text" Width="99%"></asp:TextBox> <asp:TextBox ID="AdvDayNotesTB" runat="server" TextMode="MultiLine" CssClass="text" Width="99%" /> The HTML is rendered as... <TEXTAREA style="WIDTH: 99%" id="ctl00_C_FV1_AdvDayNotesTB" rows="1" cols="20">
Does SQL Server 2008 have a built in way to log all inserts, updates and deletes on a table?The updates, inserts, deletes would come from an asp.net 3.5 webpage using SQLDataSources.If so how would I attach the UserID of the website to the database changes logs?
I have a Customer object that has two address ID fields (one for HomeAddress and one for WorkAddress)
I am loading the Address Objects by using Include="HomeAddress, WorkAddress" and then binding to the properties using the Navigation Property on the Customer Entity
Can I use just one FormView for doing the Insert/Updates and binding it to an EntityDataSource or would I have to do all the Inserting/Updating manually?
