DataSource Controls :: Datatable To Excel Memory Exception?
Feb 1, 2010[Code]....
View 4 Replies[Code]....
View 4 RepliesWe are using below code to export gridview data to excel. It is working fine when export 2 or 3 months data with more than 120,000 rows but we can try to export data for more than 12 months which almost contain 480,000 rows so it generates error
"System is out of memory exception" ....
make a query which will give me 200,000 results and bind it to the datatable but the error will pop out saying out of memory or request timed out.Can datatable/dataset handle these records or is my current server's hardware needs a little upgrade?Actually we are gettings millions of data and i dont have any idea how to solve this one? Have you worked on this kind of data? Do you have any idea on how to send my command from server A to server B while server A also processing request and will not wait for server B's result and server B will just update the records in my database, i think this can solve my problem if doing it in parallel.
View 1 RepliesI have an application which was recently ported from Classic ASP to .NET 3.5. I'm pulling some data from a stored procedure and once retrieved, I'm doing using the Select method on the DataTable in a loop so I don't have to keep going back to the database.It appears that there's something wrong here as I can refresh the page repeatedly and see the memory usage increase by a few megabytes, up to the point where we get an error. If we recycle the app pool in IIS, it starts working again, for a little while.
Is there a memory leak in this method? I did some research on the subject and Microsoft said there was an issue like that in .NET 1.0 and 1.1, but I'm using a higher version.Or, is this something else, such as not managing resources correctly? Here's a snipped of my code:
[Code]....
Initially, this app was using XML instead of a datatable and using XPATH to query the data, but it was too slow. I thought using a dataset with DataTable selects was a better option, but now we found this issue. I was thinking since the loop executes several hundred times that getting all the data and filtering through it on the client was a better choice instead of calling several web methods to get the subsets of data.
it is apparent that "string builder" is better for building large string compared to sMyString = sMyString + "something";
what is a better way to search large in memory datasets? (like the example above - i know i could use loops, is there another way that i dont now about?)
Does dataset store all its datatables' records into server memory if I just call one datatable from it?Lets make an example, a dataset with three datatables - tbCompany, tbCompanyStaffs, tbSalesOrderwhere tbCompany is the parent of tbCompanyStaffs. tbSalesOrder is an independant table.Does dataset load all tbCompanyStaffs records into the server memory as well when a class retrive data from tbCompany?
View 3 RepliesI get a dataset from the OleDbDataAdapter for a query.I need to modify some the contents in the datatable.The first column is a date field which repeats. I want to delete the duplicates so that it looks like a group by column.I tried the following
1) Trying to modify the dataset looping through it. I get an error that dataset is read only
2) Cloned the dataset and tried to modify that rows. I get an error Column 'Entered_Date' is read only
I have a memory leak somewhere that I cant find. Every few days my server will crash, and just before that I log a ton of SQL errors stating that it is "out of memory".
I cant find it anywhere, all of my connections are being disposed like so:
[Code]....
Then I call the connection from my pages like so:[Code]....
That is all pretty straight forward. The connection is disposed because it is implementing the USING clause. I am opening the connection in my connection manager class, and not where it is being utilized?Or, could the problem be in the below method I am using to populate a SqlDataReader:[Code]....
Now, at first it appears as though this could be the problem because the Connection isn't part of a USING clause, however doesn't the 'Data.CommandBehavior.CloseConnection' pretty much do the same thing. This makes sure that the connection is closed when the reader is closed, right? Here is how I call that above reader from my login page code behind:[Code]....
So the DataReader will get closed even without the .Close() because it is in the USING, and the connection should get closed because I specified it in the ExecuteReader paramters right?
I am using the below code to Export DataTable to Word,Excel,CSV format & it's working fine. But problem is that this code export to MS Word 2003,Excel 2003 & CSV version. I need to Export my DataTable to MS Word 2007,Excel 2007 & CSV because I am supposed to handle more than 100,000 records at a time and as we know Excel 2003 supports for only 65,000 records.
how to export DataTable or DataSet to MS Word 2007,Excel 2007 & CSV.
[Code]....
I am working on a ASP.NET 2.0 application. I get the following exception sometimes when i access the web application. I believe it is related to the server where the application is hosted? Exception of type 'System.OutOfMemoryException' was thrown. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. Source Error: An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Stack Trace:
[OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.]
System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection) +0
System.Reflection.Assembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection) +43
System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) +127
System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) +142
System.Reflection.Assembly.Load(String assemblyString) +28
System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective) +46
[ConfigurationErrorsException: Exception of type 'System.OutOfMemoryException' was thrown.]
System.Web.Configuration.CompilationSection.LoadAssemblyHelper(String assemblyName, Boolean starDirective) +613
System.Web.Configuration.CompilationSection.LoadAllAssembliesFromAppDomainBinDirectory() +203
System.Web.Configuration.CompilationSection.LoadAssembly(AssemblyInfo ai) +105
System.Web.Compilation.BuildManager.GetReferencedAssemblies(CompilationSection compConfig) +178
System.Web.Compilation.BuildProvidersCompiler..ctor(VirtualPath configPath, Boolean supportLocalization, String outputAssemblyName) +54
System.Web.Compilation.ApplicationBuildProvider.GetGlobalAsaxBuildResult(Boolean isPrecompiledApp) +227
System.Web.Compilation.BuildManager.CompileGlobalAsax() +52
System.Web.Compilation.BuildManager.EnsureTopLevelFilesCompiled() +337
[HttpException (0x80004005): Exception of type 'System.OutOfMemoryException' was thrown.]
System.Web.Compilation.BuildManager.ReportTopLevelCompilationException() +58
System.Web.Compilation.BuildManager.EnsureTopLevelFilesCompiled() +512
System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters) +729
[HttpException (0x80004005): Exception of type 'System.OutOfMemoryException' was thrown.]
System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +8890735
System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +85
System.Web.HttpRuntime.ProcessRequestInternal(HttpWorkerRequest wr) +259
What can be the reason for this error and what to check on the code or the server?
i need to restore Database.mdf; I create a blank new database exactly the same name as the .mdf file. However, I could not restore the database.
The error message prompted was:
TITLE: Microsoft SQL Server Management Studio Express
An exception occurred while executing a Transact-SQL statement or batch.
(Microsoft.SqlServer.Express.ConnectionInfo)
ADDITIONAL INFORMATION:
Cannot open backup device 'C:inetpubwwwrootTCPSystemApp_DataDatabase.mdf'. Operating system error 32(error not found).
RESTORE HEADERONLY is terminating abnormally. (Microsoft SQL Server, Error: 3201)
click: [URL]
BUTTONS:
OK
i have the dataset with one table.Table contains three column like 'Name','Location','Pin'.I would like to move the data from another table based on schema.
View 2 RepliesI have a procedure where I am building a table to store rows of data. I did not think this was to memory intensive. However my form does not seem load effortlessly. I call this procedure in the Form Load Event. Is there a more optimal place (Form Loading Event Procedure) to put code that may require a little more memory than usual?
View 7 RepliesHow to isolate an out of memory exception
View 2 RepliesI have a datatable with morethan 100o records and around 6 to 7 columns. When I get the data into datatable, I use each field to get distinct values and display in different drodownlists. When all the dropdown lists are filled with data, user selects one of them to filter other dropdown lists with the selected value. I want to know how can I keep datatable data stored till I get all the filtering done. I dont want to store the datatable in session as it may occupy so much of memory. Is there any other way to store the datatable till the end of the process.
View 8 RepliesI have a large XML file that contains invalid hex characters. I'm trying to test a function that removes invalid characters from the XML file but it must be stored in a string. When I try to read the XML file into a string I receive an out of memory exception. The file is 149MB. Can anyone tell me if there is another way to do this?
Code:
[code]....
I have a strange problem with ASP.NET. Several sites on my web server run the exact same application. The application has a page that contains a Tab Container from the Ajax Control Toolkit which in itself contains a Calendar control.
If I set my culture in the web.config to be:
<globalization culture="en-GB" uiCulture="en-GB" />
Then everything works fine. However, by changing this to:
<globalization culture="en-US" uiCulture="en-US" />
When attempting to load the page I get an out of memory exception and w3wp.exe seems to use all available memory.
I'm not sure why but on random occassions after I have added an update to my site via ftp and then try to access the site through the browser I get an "out of memory" exception. But there are no infinite loops or problems like that. Then after some time it will then work (I assume the web hosts restart IIS or something) and all is fine again. Server Error in '/' Application. Exception of type 'System.OutOfMemoryException' was thrown. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
Source Error:
[Code]....
Stack Trace:
[Code]....
I've got an asp.net application, it's running on DotNetNuke, under load we get the occasional out of memory exception. I've got a dump loaded it into windbg. the end of !dumpheap -stat is
1192a588 88684 2128416 AutoMapper.MappingEngine
79333594 9482 2266348 System.Byte[]
134b0034 88695 2838240 System.EventHandler`1[[AutoMapper.TypeMapCreatedEventArgs, AutoMapper]]
13d8703c 88684 4611568 System.Collections.Generic.Dictionary`2[[Castle.DynamicProxy.Generators.CacheKey, AutoMapper],[System.Type, mscorlib]]
13d86dc8 88684 4611568 Castle.DynamicProxy.ModuleScope
13d865bc 88684 4611568 System.Collections.Generic.Dictionary`2[[AutoMapper.Internal.TypePair, AutoMapper],[AutoMapper.IObjectMapper, AutoMapper]]
79327434 88703 4612556 System.Collections.Generic.Dictionary`2[[System.String, mscorlib],[System.Int32, mscorlib]]
6c38e4e4 88684 5675776 System.Threading.ReaderWriterLockSlim
79330b24 87736 6458012 System.String
000d9c88 129 24186884 Free
793042f4 221202 101117016 System.Object[]
6c38e4a0 22703104 544874496 System.Threading.ReaderWriterCount
I can't find much information on the System.Threading.ReaderWriterCount, as it seems to be the problem. What is the likely cause? Or failing that what's the best next step to work that out? Based on the pointer from the given answer I had a look at ReaderWriterLockSlim. I wasn't using it directly, but I saw that it had 88684 instances, digging deeper I saw quite a few classes with that number of instances, pointing to AutoMapper.MappingEngine. This should be a singleton, so I've had a look at where it's being created. I suspect that it's the DI container and have made some changes around that to see if it helps
I have a requirement that to create a image with 300DPI, but during the composition of image, like addding the resources (font, images) I am getting out of memory exception.
using (Bitmap pg = new Bitmap(GetPixelsFromInches(float.Parse(pageWidth), dpi, actualDpi), GetPixelsFromInches(float.Parse(pageHeight), dpi, actualDpi)))
{
pg.SetResolution(float.Parse(dpi), float.Parse(dpi));
Graphics gr = Graphics.FromImage(pg);
gr.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
System.Drawing.Rectangle pgRect = new System.Drawing.Rectangle(0, 0, pg.Width, pg.Height);
SolidBrush solidWhite = new SolidBrush(Color.White);
gr.FillRectangle(solidWhite, pgRect);
currentPageDisplayed = xNode.Attributes["id"].Value;
foreach (XmlElement xElement in xNode)
{
//Here I am writing each elements, like texts or images.
DrawImageForElements(xElement, , dpi, actualDpi);
}
MemoryStream myMemoryStream = new MemoryStream();
//pg.Save(myMemoryStream, System.Drawing.Imaging.ImageFormat.Bmp);
String filename=@"c:images" + currentPageDisplayed + "+.png";
pg.Save(filename, System.Drawing.Imaging.ImageFormat.Png);
myMemoryStream.Dispose();
gr.Dispose();
pg.Dispose();
}
Im trying to retrieve a datatable that I put into a session variable, however when i do it apears that there are no columns... I have done this before in VB.NET, I'n now using C#, and it worked perfectly, and as far as i can see there is no change in the code other than the obvious syntax changes.
EDIT: The dt (in part 2) variable has the right data when i look in the data visulization window, but i have noticed that the other properties associated with a datatable are abscent. When i hover over a normal looking datatable with my mouse it looks like the following: " dt ----- {System.Data.DataTable}" but in the class im working on it just looks like "dt ----- {}". Also, after I return the session variable into the dt I clone it (dtclone = dt.clone(); ) and the clone is empty in the data visulizer.... what on earth!
EDIT 2: I have now also tried converting the first datatable to a dataset, putting this in the session variable, and recoverting it back to a datatable in the class. Am starting to wonder if it is a probelm with: dt.Load(sqlReader); The data does appear after this step though in the dataset visualiser, but not after being cloned. Code below.
1) SQL command in a webhandler, the results of which populate the datatable to be put into the session variable.
DataTable dt = new DataTable();
SqlDataReader sqlReader= default(SqlDataReader);
SqlDataAdapter sqlAdapter = new SqlDataAdapter();
sqlReader = storedProc.ExecuteReader(CommandBehavior.CloseConnection);
dt.Load(sqlReader);
System.Web.HttpContext.Current.Session["ResultsTable"] = dt;
2) Part of the code in a class which performs calculations on the table:
DataTable dt = (DataTable)HttpContext.Current.Session["ResultsTable"];
these below codes give whole data of my Rehber datas. But if i want to show web page via Gridview send me out of memory exception error.
[code]....
im using sap crystal report version 13.0.2000.0. i've 27,000 records to be printed but when i run my application the exception error is displayed "Out Of Memory".. i've my appliation in VS2010 and OS is server 64bit and 4GB RAM..
View 1 RepliesI'm using SqlBulkCopy to load large file into databse, but a file bigger than 200k give me an error outofmemoryexception. Is there any way to append data into database from a file splitted in two. the first time I can use SqlBulkCopy, but to append second file into my table.
View 1 RepliesI have two datatables, I want to get all records which are in one datatable but all matching records which exist in another datatable should not be available.
In terms of set, you can say that record which are in 1st datatable but not in 2nd DataTable.