Azure Data Factory Copy Activity


have been working on couple days , cannot past error. have 2 activities in pipeline. first activity copies data odbc connection azure database, successful. 2nd activity transfers data azure table azure table , keeps failing.

error message is: copy activity met invalid parameters: 'unknownparametername', detailed message: item same key has been added..

not see invalid parameters or unknown parameter names. have rewritten multiple times using add activity code template , myself, not receive errors when deploying on when running. below json pipeline code.

2nd activity receiving error.

thanks.

source data set

{ "name": "analyticsdb-shipups_06shp-01src_az-915pm", "properties": {     "structure": [         {             "name": "upsd_bol",             "type": "string"         },         {             "name": "upsd_ordn",             "type": "string"         }     ],     "published": false,     "type": "azuresqltable",     "linkedservicename": "source-sqlazure",     "typeproperties": {},     "availability": {         "frequency": "day",         "interval": 1,         "offset": "04:15:00"     },     "external": true,     "policy": {} }

destination data set

{ "name": "analyticsdb-shipups_06shp-02dst_az-915pm", "properties": {     "structure": [         {             "name": "ship_sys_track_num",             "type": "string"         },         {             "name": "ship_track_num",             "type": "string"         }     ],     "published": false,     "type": "azuresqltable",     "linkedservicename": "destination-azure-analyticsdb",     "typeproperties": {         "tablename": "[olcm].[ship_tracking]"     },     "availability": {         "frequency": "day",         "interval": 1,         "offset": "04:15:00"     },     "external": false,     "policy": {} }

pipeline

 { "name": "shipups_fc_copy-915pm", "properties": {     "description": "copy shipments ",     "activities": [         {             "type": "copy",             "typeproperties": {                 "source": {                     "type": "relationalsource",                     "query": "$$text.format('select company, upsd_ordn, upsd_bol \"orupsd - ups interface dtl\" company = \\'01\\'', windowstart, windowend)"                 },                 "sink": {                     "type": "sqlsink",                     "sqlwritercleanupscript": "$$text.format('delete imp_fc.ship_ups_intdtl_tracking', windowstart, windowend)",                     "writebatchsize": 0,                     "writebatchtimeout": "00:00:00"                 },                 "translator": {                     "type": "tabulartranslator",                     "columnmappings": "company:company, upsd_ordn:upsd_ordn, upsd_bol:upsd_bol"                 }             },             "inputs": [                 {                     "name": "analyticsdb-shipups_03shp-01src_fc-915pm"                 }             ],             "outputs": [                 {                     "name": "analyticsdb-shipups_03shp-02dst_az-915pm"                 }             ],             "policy": {                 "timeout": "1.00:00:00",                 "concurrency": 1,                 "executionpriorityorder": "newestfirst",                 "style": "startofinterval",                 "retry": 3,                 "longretry": 0,                 "longretryinterval": "00:00:00"             },             "scheduler": {                 "frequency": "day",                 "interval": 1,                 "offset": "04:15:00"             },             "name": "915pm-shipups-fc-copy->[imp_fc]_[ship_ups_intdtl_tracking]"         },         {             "type": "copy",             "typeproperties": {                 "source": {                     "type": "sqlsource",                     "sqlreaderquery": "$$text.format('select distinct ups.upsd_bol, ups.upsd_bol imp_fc.ship_ups_intdtl_tracking ups left join olcm.ship_tracking st on ups.upsd_bol = st.ship_sys_track_num st.ship_sys_track_num null', windowstart, windowend)"                 },                 "sink": {                     "type": "sqlsink",                     "writebatchsize": 0,                     "writebatchtimeout": "00:00:00"                 },                 "translator": {                     "type": "tabulartranslator",                     "columnmappings": "upsd_bol:ship_sys_track_num, upsd_bol:ship_track_num"                 }             },             "inputs": [                 {                     "name": "analyticsdb-shipups_06shp-01src_az-915pm"                 }             ],             "outputs": [                 {                     "name": "analyticsdb-shipups_06shp-02dst_az-915pm"                 }             ],             "policy": {                 "timeout": "1.00:00:00",                 "concurrency": 1,                 "executionpriorityorder": "newestfirst",                 "style": "startofinterval",                 "retry": 3,                 "longretryinterval": "00:00:00"             },             "scheduler": {                 "frequency": "day",                 "interval": 1,                 "offset": "04:15:00"             },             "name": "915pm-shipups-az-update->[olcm]_[ship_tracking]"         }     ],     "start": "2017-08-22t03:00:00z",     "end": "2099-12-31t08:00:00z",     "ispaused": false,     "hubname": "adf-tm-prod-01_hub",     "pipelinemode": "scheduled" } }






Microsoft Azure  >  Azure Data Factory



Comments

Post a Comment

Popular posts from this blog

Azure DocumentDB Owner resource does not exist

having same error microsoft.azure.documents.documentclientexception: message: {"errors":["owner resource not exist"]} , scenario. when deployed webapp azure , try document docdb throws error. docdb exists in azure , contains document looking for. weird local machine(running vs) works fine. i'm using same settings in azure , local. have , idea this. thanks Microsoft Azure  >  Azure Cosmos DB
Read more

RFC_ERROR_SYSTEM_FAILURE with SAP ECC 6 Unicode

hi all, i using biztalk 2006 r2, microsoft biztalk adapter v2.0 mysap business suite sp1 & sql server 2005 database server. we have developed orchestrations communicating sap version 4.7.   currently sap has been upgraded in ecc 6(unicode) system. i have configured new send port on dev server. orchestration can communicate sap ecc 6.0 (non- unicode) system while trying connect sap ecc 6.0 (unicode) getting error. a message sent adapter "sap" on send port "ecc 6 sendport" uri "sap://as:***.**.***.***/**/***/" suspended.   error details: rfc_error_system_failure   messageid:   {b7dd677e-0ef3-480b-835a-d389f2e24179} is there can configure setting unicode ?   thanks in advance     ajay dabhade problem solved: standard table data types appear not unicode compatible in sap ecc 6.0. make rfcgetfunctiondesc raise error. solution: used other data types scratch in rfcs in sap   ajay dabhad...
Read more

C# System.Data.Common DbCommand and getting Datasets from Oracle

i found no oracle specific forum area, decided post here. getting dataset oracle c# system.data.common library's dbcommand seems tricky task. i using command object in project along dbproviderfactory , dbdataadapter because meant allow use same library multiple database providers, , don't have clutter data access layer of project separate classes accessing each individual database provider. my problem is, stored procedures fetching data oracle require refcursor object added in parameters. however, dbtype library not contain definition object. using "object"-type not work. i have read many places not have add refcursor parameter, endless failed attempts @ running command without refcursor object has brought me conclusion leaving out refcursor object not option when using system.data.common library. here coded example connectionstring purposefully left out.                 dbconnection.ope...
Read more