DataSource Controls :: SSIS: RetainSameConnection And Variable Locking?
Jun 8, 2010
I am modifying a rather large ETL package using BIDS. It builds a datawarehouse, so there are many sources and a single destination. There are quite a few data flow tasks, and their connection strings are read into variables from an encrypted source. All the data flow tasks are arranged serially because of variable locking issues - occasionally two DFTs attempt to simultaneously lock the variable and it fails. I realize there are script solutions out there, but I am looking for perhaps a more elegant (easier) solution than to apply scripted solutions to tons of packages over something so trivial.
I came across the "RetainSameConnection" connection property, and I have a hunch this could be the solution. My reasoning is: If there's only one connection to a data source, there should only be one read operation from the variable. At least, it makes sense when SOURCING.As for destinations, I'm not sure if it's a good idea to have data from multiple sources flowing in through a single connection.This is my first week working in ETL so I don't know all the nuances of databases, ETLs, connections, etc.
I'm using a 3rd party web service in my ASP.NET application. Calls to the 3rd party web service have to be synchronized, but ASP.NET is obviously multi-threaded and multiple page requests could be made that result in simultaneous calls to the 3rd party web service. Calls to the web service are encapsulated in a custom object. My thought is to store the object in an application variable and use the C# lock keyword to force synchronized use of it.
I'm nervous, because I'm new to multi threaded concepts and I've read that you shouldn't lock a public object (which my application variable effectively is). I've also read that if the locked block of code fails (which it could if the web service fails), then it could destabilize the app domain and bring down the application.I should mention that the 3rd party web service is rarely used in my website and it's going to be rare that 2 requests to it are made at the same time.Here's a rough code sample of how I'd make calls to the web service:
I need to have maybe 5 threads running which all use the same DataSet and need to loop through all the rows retrieved but each row must be processed by one thread only (e.g. I don't want row 1 to be processed by thread 1 and thread 3).
I can also use a DataReader if that helps, but I need to dow some locking logic I would have thought.
In the development environment, in SSMSE > MyDatabase > Security > MyUser > Database role membership I have checked db_owner to allow me to build the database.
Now I am copying the database to the production environment I would like to lock the security down tightly. In the past (with classic ASP applications) I have only checked db_datawriter and db_datareader and, in Securables, checked Execute Grant on the stored procedures.
In Database > Securables I add my stored procedures and check Execute/Grant for each of them and click OK. When I return to the securables page the stored procedures are no longer listed.
I have noticed that nothing is listed in Effective Permissions and I think it should list Execute. If this is the problem, how do I add Execute to the Effective Permissions list?
I have a web project that sends out a customized email after certain user events (like a status change). I also have an SSIS package on the database that pulls in data from one database and updates another database with that data. I want to be able to compare the data and if a particular field's data for that record is different between the two databases is different, I want to be able to use the dll I created for the web app to send the appropriate email.
What's the best approach for loading a flat flat according to its name? A Mainframe job places a file on a server for me and it always beings with MISA and I load the file monthly. After loading the file I'd like to move it into a History folder on my server.Should I use a Script Task or the Foor Loop Container to accomplish this? Keep in mind I only want to load files that begin with "MISA".
i am having a problem in executing the SSIS package through an asp.net application. i'm using c#. every time i run the package through web, i think the package is not disposed properly so until and unless i stop the application the package won't execute...i am not able to dispose or unload the package. i'm geeting errors like
'WebDev.WebServer.EXE' (Managed): Loaded 'C:WINDOWSassemblyGAC_32CustomMarshalers2.0.0.0__b03f5f7f11d50a3aCustomMarshalers.dll', Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled. The result of the expression ""EXEC " + @[User::vRowData]" on property "SqlStatementSource" cannot be written to the property.
The expression was evaluated, but cannot be set on the property. : Package Execution results: {0}
The short story is, I'm developing an SSIS package to select from a remote Oracle database (read: no Windows Authentication) and store into an SQL Server db. I can execute the package just fine in development and after importing, but when I run it as a scheduled job, I get an ora-01017 error (bad un/pw). The details are, I built this package and tested in in BIDS but came across the problem that BIDS really doesn't save the password I'd set for the Oracle connection. After poking around a bit, I found this
[URL] which showed how to create an XML configuration which the package could then read. Fiddled around with that, got it working in BIDS. Connected to Integration Services on the server on which I want to run the package, imported it, right-clicked and did the "Run Package" option as a test, it works great. However, when I connect to the same server's Database Engine to create and schedule a job, I cannot connect to Oracle. For some reason, it seems as if the package is not reading the configuration when run in schedule.
I've tried removing the job and re-working it, thinking maybe it didn't pick up the new version when I re-imported the package. A new job had the same problem. Next, I tried re-naming the .dtsConfig and doing the "Run Package" option as a test. Failed with the same error. A bit surprised, I thought I'd get some kind of file error and we wouldn't get as far as trying to talk to Oracle. Anyway, thinking maybe it's permissions on the configuration, fixed the .dtsConfig name and gave full access to Everyone. That ought to let the Agent see it. Scheduled up the job, still can't talk to Oracle. So it seems as if the scheduled version is ignoring the configuration. Anyone have any idea what I'm missing, that scheduling an SSIS package makes it act differently than running it manually? BTW, the instance under which we're running the package is VS 2005. BIDS 2005 to build the package.
In my initial cycle, I put the package configuration on a shared drive. This works great for my dev box and the dev db box, as they have the drive mapped in the same way.Unfortunately, the proposed production server has E mapped elsewhere. "Easy," I thought, "I'll just change to use the UNC version of that path."When I attempt to run the package on my development box, I get an error in BIDS that says it can't copy to the deployment utility output directory.This is wrong, as I delete the file, start the package, and it magically appears in my deployment/bin directory. Itdoes, in fact, copy the file, then has this error.I changed my reference back to the mapped drive, and the build works.
I have a number of existing SSIS packages on SQL Server 2008 and want to deploy them to another server. What is the easiest way to deploy multiple packages to another server?
Previous developers have developed a .SSISDeploymentManifest file (SQL Server 2005) to install SSIS packages which prompts a Package Installation wizard. I tried to install it on SQL Sever 2008 (it works on SQL Server 2005), but it failed with the error: "Storing or modifying packages in SQL Server requires the SSIS runtime and database to be the same version. Storing packages in earlier versions is not supported".
How do I install 2005 SSIS packages on SQL Server 2008? I have all the .dtsx files, not sure how to install them?
As you can see, the connectionstring parameter is defined to a specific connection string name and I need to be able to set such a parameter to a different value, for example, to a session variable content.
I am storing a custom "Organisation" object as a session variable. One of the properties of the Organisation object is "OrganisationID" (integer). I have a DataSource that requires a parameter value to run, and I want to use a SessionParameter to populate this. In a previous version, I stored the OrganisationID directly as a session variable. In that case, I could easily access it like this:
[Code]....
However, how do I now access the OrganisationID property of an "Organisation" type session variable (called "Organisation")? I have tried this, which does not seem to work: <asp:SessionParameter Name="OrganisationID" SessionField="Organisation.OrganisationID" Type="Int32" />
Error: Executed as user: MACSTEELUSA.COMsa. ...9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 8:00:17 PM Error: 2010-06-02 20:00:18.56 Code: 0xC0202009 Source: CRM_ORACLE_ARSUMMARY Connection manager "SourceConnectionOLEDB" Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft OLE DB Provider for Oracle" Hresult: 0x80004005 Description: "Oracle client and networking components were not found. These components are supplied by Oracle Corporation and are part of the Oracle Version 7.3.3 or later client software installation. Provider is unable to function until these components are installed.". End Error Error: 2010-06-02 20:00:18.58 Code: 0xC020801C Source: Data Flow Task Source - Query [1] Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireCon... The package execution fa... The step failed.
Tasks Taken:Went to this folder checked for tnsnames.ora file but it is in good shape.
Is it possible to use a database name as a variable in a SP?I use a separate database for each custromer.I also use a master database that keeps database name, ID, and Password.I then want to generate an SQL select against the appropriate Database based on the userID and password (for an XML REsponse)is this possible?Set @DBName as varchar(50)Select * from @DBName.dbo.tablename where bla bla blaNote: that the actual select is about 150 lines with multiple sub selects
Would anyone know how to pass a variable to SqlDataAdapter. Here is my code.
TextBox1.Text = 1 Dim textboxval As String textboxval = TextBox1.Text connection.Open()Using connection As New SqlConnection("Data Source=CHRIS-PCSQLEXPRESS;Initial Catalog=TorGHL;Integrated Security=True") Dim dapubcb As New SqlDataAdapter("SELECT [BarName], [MenuSubId], [Address], [PhoneNo], [BarURL], [Email] [BarDescription], [Id], [bg_long], [bg_lat] FROM [BarDetails] WHERE [MenuSubId] = texboxval ", connection)
I have an objectDataSource which uses my stored procedure GetNotifications
And my GridView is bound to that objectDataSource
the only problem is that my stored procedure requires the variable @UserName and then returns the values for that username, but I do not know how to pass the variable to the objectDataSource, does anyone know the code for this?
I can get userName from HttpContext.Current.User.Identity.Name but how do I pass that to the objectDataSource?
I am trying to pass a variable within a selectcommand statment in ASP.NET and I have researched online how to do it but I can't seem to find a way to do it...What I am trying to do is write a code for a web page that will display a page with useful links on it where the links are separated into different topics and where everything is dynamically grabbed from a database. This is what I have so far:
declare @stateQuery as nvarchar(200) if @stateid is null set @stateQuery='' if @stateid is not null set @stateQuery='and tbl_city.stateid ='+cast(@stateid as nvarchar(10)) set @length=(select top 1 len(dbo.Tbl_city.cityName) FROM dbo.tbl_city INNER JOIN tbl_cityinfonew ON dbo.tbl_city.cityId = tbl_cityinfonew.cityId where (tbl_cityinfonew.language =@language )+cast( @stateQuery as nvarchar(100)) order by tbl_cityinfonew.createdate desc)
