DataSource Controls :: Datatable/Dataset Total Memory?
Mar 16, 2011
make a query which will give me 200,000 results and bind it to the datatable but the error will pop out saying out of memory or request timed out.Can datatable/dataset handle these records or is my current server's hardware needs a little upgrade?Actually we are gettings millions of data and i dont have any idea how to solve this one? Have you worked on this kind of data? Do you have any idea on how to send my command from server A to server B while server A also processing request and will not wait for server B's result and server B will just update the records in my database, i think this can solve my problem if doing it in parallel.
it is apparent that "string builder" is better for building large string compared to sMyString = sMyString + "something";
what is a better way to search large in memory datasets? (like the example above - i know i could use loops, is there another way that i dont now about?)
Does dataset store all its datatables' records into server memory if I just call one datatable from it?Lets make an example, a dataset with three datatables - tbCompany, tbCompanyStaffs, tbSalesOrderwhere tbCompany is the parent of tbCompanyStaffs. tbSalesOrder is an independant table.Does dataset load all tbCompanyStaffs records into the server memory as well when a class retrive data from tbCompany?
I have an application which was recently ported from Classic ASP to .NET 3.5. I'm pulling some data from a stored procedure and once retrieved, I'm doing using the Select method on the DataTable in a loop so I don't have to keep going back to the database.It appears that there's something wrong here as I can refresh the page repeatedly and see the memory usage increase by a few megabytes, up to the point where we get an error. If we recycle the app pool in IIS, it starts working again, for a little while.
Is there a memory leak in this method? I did some research on the subject and Microsoft said there was an issue like that in .NET 1.0 and 1.1, but I'm using a higher version.Or, is this something else, such as not managing resources correctly? Here's a snipped of my code:
[Code]....
Initially, this app was using XML instead of a datatable and using XPATH to query the data, but it was too slow. I thought using a dataset with DataTable selects was a better option, but now we found this issue. I was thinking since the loop executes several hundred times that getting all the data and filtering through it on the client was a better choice instead of calling several web methods to get the subsets of data.
I'm returning one table. My understanding is that a DataSet can be made of more than one DataTable. I also understand that I can use either of these to return my one table.
Is it better to use the DataTable when I have one table to return instead of a DataSet? Is there any advantage of not using the DataSet in a situation like this? Is there a disadvantage to using the DataSet when only one table is being returned? Does this even matter?
I need to create an interface for adding users to .Net Membership DB through an existing database. My Idea is to read each row from database from existing users table in a datatable and then iterate through each row and add it to new database. I cannot directly copy the rows from existing users table because the password stored in .Net Membership provider is in ir-reversible hash code and hashing is done through encryption algorithm present in front end code. My User count is huge (in millions).So what should i use to read the records, Dataset or datatable. Also tell what is the max number of rows it can hold.
I have a DataSet with multiple DataTables and I use an ObjectDataDataSource to bind my DataGrid to it. One of my tables is correctly displayed in the grid. My problem is, that the grid displays the wrong table.
My tables are related to each other - I make use of the "relation" objects in VS Studio 2010. My ObjectDataSource points to a class with a select method. The DataSet contains several DataTables (e.g. "Variables" and "Components").
[Code]....
How can I configure the DataSet to display the table "Variables" by default?
I am having problem with calculated columns in the following code
[Code]....
orderDT is the DataTable. When "Extra" column has nothing (I am NOT using Default), "Total" comes out to be blank? "Price" will have a value (always) "Extra" may not have any value
I have a memory leak somewhere that I cant find. Every few days my server will crash, and just before that I log a ton of SQL errors stating that it is "out of memory".
I cant find it anywhere, all of my connections are being disposed like so:
[Code].... Then I call the connection from my pages like so:[Code]....
That is all pretty straight forward. The connection is disposed because it is implementing the USING clause. I am opening the connection in my connection manager class, and not where it is being utilized?Or, could the problem be in the below method I am using to populate a SqlDataReader:[Code]....
Now, at first it appears as though this could be the problem because the Connection isn't part of a USING clause, however doesn't the 'Data.CommandBehavior.CloseConnection' pretty much do the same thing. This makes sure that the connection is closed when the reader is closed, right? Here is how I call that above reader from my login page code behind:[Code]....
So the DataReader will get closed even without the .Close() because it is in the USING, and the connection should get closed because I specified it in the ExecuteReader paramters right?
i have the dataset with one table.Table contains three column like 'Name','Location','Pin'.I would like to move the data from another table based on schema.
I have a procedure where I am building a table to store rows of data. I did not think this was to memory intensive. However my form does not seem load effortlessly. I call this procedure in the Form Load Event. Is there a more optimal place (Form Loading Event Procedure) to put code that may require a little more memory than usual?
how do i copy required row in dataset to a new dataset. i have bind a xml into a dataset and i need to display say row 3 to 5 only so i do a for loop but i have encounter problem when trying to copy the rows to a new dataset.
do i have to write the xml to datatable and bind it to dataset and from there copy over ?
I have a datatable with morethan 100o records and around 6 to 7 columns. When I get the data into datatable, I use each field to get distinct values and display in different drodownlists. When all the dropdown lists are filled with data, user selects one of them to filter other dropdown lists with the selected value. I want to know how can I keep datatable data stored till I get all the filtering done. I dont want to store the datatable in session as it may occupy so much of memory. Is there any other way to store the datatable till the end of the process.
I need your help / guidence for doing custom paging in c#, through sqlserver. so that i can have data in dataset/datatable according to the pagesize given, not all the data in dataset. some code /url .......with full discrption if possible.
Im trying to retrieve a datatable that I put into a session variable, however when i do it apears that there are no columns... I have done this before in VB.NET, I'n now using C#, and it worked perfectly, and as far as i can see there is no change in the code other than the obvious syntax changes.
EDIT: The dt (in part 2) variable has the right data when i look in the data visulization window, but i have noticed that the other properties associated with a datatable are abscent. When i hover over a normal looking datatable with my mouse it looks like the following: " dt ----- {System.Data.DataTable}" but in the class im working on it just looks like "dt ----- {}". Also, after I return the session variable into the dt I clone it (dtclone = dt.clone(); ) and the clone is empty in the data visulizer.... what on earth!
EDIT 2: I have now also tried converting the first datatable to a dataset, putting this in the session variable, and recoverting it back to a datatable in the class. Am starting to wonder if it is a probelm with: dt.Load(sqlReader); The data does appear after this step though in the dataset visualiser, but not after being cloned. Code below.
1) SQL command in a webhandler, the results of which populate the datatable to be put into the session variable.
DataTable dt = new DataTable(); SqlDataReader sqlReader= default(SqlDataReader); SqlDataAdapter sqlAdapter = new SqlDataAdapter(); sqlReader = storedProc.ExecuteReader(CommandBehavior.CloseConnection); dt.Load(sqlReader); System.Web.HttpContext.Current.Session["ResultsTable"] = dt;
2) Part of the code in a class which performs calculations on the table:
I have two datatables, I want to get all records which are in one datatable but all matching records which exist in another datatable should not be available.
In terms of set, you can say that record which are in 1st datatable but not in 2nd DataTable.