Reading - Parsing And Posting Data From A .csv File Into A SQL Database
Jul 13, 2010
I have the following data that basically I only need a few bits of information from:
Resource:X - Y;Z - Å;Type:(all) From Date:
07/12/2010 - To Date 07/12/2010 Sort
by:Time Include Referring
source/physician:No Footer:Default
Criteria:None
","Appointments","X,
Y","ZAssociates","Monday, July 12,
2010","Time","Patient Name","Patient
ID","Appt. Type","Ref. Source/
Physician","Phone","Type","DOB
","Z, X","Y","7/12/2010
12:00:00AM","Time","Patient
Name","Patient ID","Appt.
Type","Phone","Type","DOB "," 7:30
[snip]
The only things I need from this are:
Patients Name
Drs Name
Patients Phone Number
Appt Time
Appt Date
and the rest of the information I can discard. A customer uploads this as a .csv file (even though it really isn't as you can see) and I'd like to parse the needed information and post that to my SQL database and discard the rest. I think I can do this with a dataset but I've never built that before. The fields from the customer will always be the same and the fields I will need will always be the same. Also, the date time has to be in the format of yyyy/mm/dd:hhmm and the phone number always has to have 512 as a prefix. Here is the code I currently have for my site:
Imports System.IO
Imports System.Data
Imports System.Data.SqlClient
Partial Class _Default
Inherits System.Web.UI.Page
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load
End Sub
Protected Sub Submit1_Click(ByVal sender As Object, ByVal e As System.EventArgs)
Dim SaveLocation = "\xxxWEB3wwwrootWebfile1RemindersDoug_Ancildoug.csv"
If UploadFile(SaveLocation) Then
'the file was uploaded: now try saving it to the database
SaveToDatabase(SaveLocation)
End If
End Sub
Private Function UploadFile(ByVal SavePath As String) As Boolean
Dim fileWasUploaded As Boolean = False 'indicates whether or not the file was uploaded
'Checking if the file upload control contains a file
If Not File1.PostedFile Is Nothing And File1.PostedFile.ContentLength > 0 Then
Try
'checking if it was .txt file BEFORE UPLOADING IT!
'You used to upload it first...but the file could be a virus
If File1.FileName = ("doug.csv") = False Then
'The file is not the expected type...do not upload it
'just post the validation message
message.Text = "Sorry, thats not the correct file."
message2.Text = "Please locate and upload 'doug.csv'"
Else
'The file is a .txt file
'checking to see if the file exists already
'If it does exist Deleting the existing one so that the new one can be created
If IO.File.Exists(SavePath) Then
IO.File.Delete(SavePath)
End If
'Now upload the file (save it to your server)
File1.PostedFile.SaveAs(SavePath)
'After saving it check to see if it exists
If File.Exists(SavePath) Then
'Upload was sucessful
message.Text = "Thank you for your submission."
fileWasUploaded = True
Else
'the file was not saved
message.Text = "Unable to save the file."
End If
End If
Catch Exc As Exception
'We encountered a problem
message.Text = "Your file was not in the correct format. Please contact Customer Service at xxxx-xxxx-xxxx."
End Try
Else
'No file was selected for uploading
message.Text = "Please select a file to upload."
End If
Return fileWasUploaded
End Function
Private Sub SaveToDatabase(ByVal SavePath As String)
Try
Dim sqlQueryText As String = _
"BULK INSERT dialerresults " + _
"FROM '" & SavePath & "' " + _
"WITH ( FIELDTERMINATOR = ',' , ROWTERMINATOR = '
' )"
' and bulk import the data:
'If ConfigurationManager.ConnectionStrings("Dialerresults") IsNot Nothing Then
'Dim connection As String = ConfigurationManager.ConnectionStrings("Dialerresults").ConnectionString
Dim connection As String = "data source=10.2.1.40;initial catalog=IVRDialer;uid=xxxx;password=xxxxx;"
Using con As New SqlConnection(connection)
con.Open()
' execute the bulk import
Using cmd As New SqlCommand(sqlQueryText, con)
cmd.ExecuteNonQuery()
End Using
End Using
'Else
'message.Text="ConfigurationManager.ConnectionStrings('Dialerresults') is Nothing!"
'End If
Catch ex As Exception
message.Text = "Your file was not in the correct format. Please contact Customer Service at xxxxxxx."
End Try
End Sub
End Class
I have the following data that basically I only need a few bits of information from: The only things I need from this are:
Patients Name Drs Name Patients Phone Number Appt Time Appt Date
and the rest of the information I can discard. A customer uploads this as a .csv file (even though it really isn't as you can see) and I'd like to parse the needed information and post that to my SQL database and discard the rest. I think I can do this with a dataset but I've never built that before. The fields from the customer will always be the same and the fields I will need will always be the same. Also, the date time has to be in the format of yyyy/mm/dd:hhmm and the phone number always has to have 512 as a prefix. Here is the code I currently have for my site:
I know this is probably a pretty easy thing to do and it is if I can upload the file and store it onto the hard drive of the server. What I need to do is read the text file into memory and then parse through it one line at a time. Anyone have any code that demonstrates that?
CS1061: 'System.Data.SqlClient.SqlDataAdapter' does not contain a definition for 'Rows' and no extension method 'Rows' accepting a first argument of type 'System.Data.SqlClient.SqlDataAdapter' could be found.
I have deveopmed i web site in ASP.NET and C# using Visual Studio 2008. I have a problem with performing button events twice. A user has a web page open. He click on a button (Save) and data is added to the database. He then presses F5 to update the page, and the button event is fired again adding more data to the database that should no be added. My question is how can I avoid this scenario?
I have a requirement in which i have a database table which has 4 columns Id,Name,position,city.....now i have a form in which the user will be filling in the id and then the other three columns should auto populate ....now i want a web service which will make a data connection and get the Id from the form and then query the database and get the other three columns..i am pretty new to web services ..can somebody give me some heads up on how to achieve this...let me know if i have to provide more details.
<link> <linkto>[URL]/broadcasts/index.cfm?fuseaction=usrbrd&broadcasterid=57468</linkto> <image>report_button</image> <alt>Report a Problem</alt> <target></target> </link> <linkto> tag have the url. But because of the "&" symbol in the url thats completely crashing the page. I tried using like amp; next to the & symbol but this doesn't give the correct url.
i have flat files in some direcotry. each flat file contains around 60,000 rows of data. earlier we were using excel sheets to upload data. but now the challenge is it is taking more time to upload data into DB tables. which is the best way to upload such a huge data into db tables. can we use SSIS for uploading data. if so, how can we achieve that.
I'm developing a small ASP.NET Mvc project in Mono 2.4, Ubuntu 10.10. There is an array of objects, each one of them corresponds to a certain xml file. Reading of the xmls is performed with XmlTextReader. That does not work because xml files have rare "cp866nav" encoding, which is not supported by XmlTextReader ("System.ArgumentException: Encoding name 'cp866nav' not supported"). But it works fine if encoding in xml header is changed to "cp866". I found a kind of solution which consists in initializing XmlTextReader with a StreamReader with a certain encoding instead of file name, like in the code below:
XmlTextReader reader = new XmlTextReader(new StreamReader(Server.MapPath(filename), Encoding.GetEncoding("cp866")));
The issue is that the directory which contains xml files is read only (I can not change it), so I get "System.UnauthorizedAccessException: Access to the path '' is denied.". Rather strange, because XmlTextReader initialized with a filename seems to read the files. Is there any solution, considering that program cannot modify or create files?
I would like ask you for some ideas how can I write mechanism that will be automatically read files from folder. I don't know how to schedule the write task. I want to check for new file everyday and if the new file is in the folder read it.
Now I need to load the CSV in C # and column 3 to distinguish what the operation is carried out.When the load line in column 3 contain the value 1, so will save the table 'Cats' when the value 2 and is saved in the table "Dogs".(Note: Data are separated ";")
I am looking for opinions and best coding practices.I need to get info from a database with a query such as SELECT this, that FROM MyDB (returning 2 fields). Would it be wrong to use a hashtable for temporary storage?Is there a better way to accomplish the same thing? What about if I am returning more than 2 fields, would it be best to step up to a DataSet or something?
public partial class _Default : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) GetData(); [code]...
when my xml file is big, I get the following error unexpected end of file while parsing name has occurred. Line 1, position 2034 the xmlData debug showed that not all the xml file is being read
[Code]....
I read that the memory of the string variable is not big enough to handle the xml file, what can I use to be able to read big xml files. my program works for small xml text.
1. I know about collections but I dont remember the precess. I know you drag a sqldatasource control on the form(Select everything in the database ie. selct *) and then delete it. But how do I use it i code.
Example. I need to select services where country=canada from the Agencies table.
2. I know how to manually pull data from an access database using the code below. I want to know how to pull data from a sql database just like the one below.
3. explain the difference between Collections and manually pulling it like the one bellow?
but when I tried to add the service reference to the silverlight app, the Add Service Reference window would show me that there was a service file, but the node wouldn't expand to show the rest of the files that contain the services and what not. I got this error instead:
"An error occurred during the processing of a configuration file required to service this request. review the specific error details below and modify your configuration file appropriately."
"Parser Error Message:There is no service behavior named 'AdventureWorks_WebServer.Service1Behavior'."
"Metadata contains a reference that cannot be resolved: 'http://localhost:55579/Service1.svc'.
The server did not provide a meaningful reply; this might be caused by a contract mismatch, a premature session shutdown or an internal server error.
If the service is defined in the current solution, try building the solution and adding the service reference again."