Azure Data Factory Copy Activity


have been working on couple days , cannot past error. have 2 activities in pipeline. first activity copies data odbc connection azure database, successful. 2nd activity transfers data azure table azure table , keeps failing.

error message is: copy activity met invalid parameters: 'unknownparametername', detailed message: item same key has been added..

not see invalid parameters or unknown parameter names. have rewritten multiple times using add activity code template , myself, not receive errors when deploying on when running. below json pipeline code.

2nd activity receiving error.

thanks.

source data set

{ "name": "analyticsdb-shipups_06shp-01src_az-915pm", "properties": {     "structure": [         {             "name": "upsd_bol",             "type": "string"         },         {             "name": "upsd_ordn",             "type": "string"         }     ],     "published": false,     "type": "azuresqltable",     "linkedservicename": "source-sqlazure",     "typeproperties": {},     "availability": {         "frequency": "day",         "interval": 1,         "offset": "04:15:00"     },     "external": true,     "policy": {} }

destination data set

{ "name": "analyticsdb-shipups_06shp-02dst_az-915pm", "properties": {     "structure": [         {             "name": "ship_sys_track_num",             "type": "string"         },         {             "name": "ship_track_num",             "type": "string"         }     ],     "published": false,     "type": "azuresqltable",     "linkedservicename": "destination-azure-analyticsdb",     "typeproperties": {         "tablename": "[olcm].[ship_tracking]"     },     "availability": {         "frequency": "day",         "interval": 1,         "offset": "04:15:00"     },     "external": false,     "policy": {} }

pipeline

 { "name": "shipups_fc_copy-915pm", "properties": {     "description": "copy shipments ",     "activities": [         {             "type": "copy",             "typeproperties": {                 "source": {                     "type": "relationalsource",                     "query": "$$text.format('select company, upsd_ordn, upsd_bol \"orupsd - ups interface dtl\" company = \\'01\\'', windowstart, windowend)"                 },                 "sink": {                     "type": "sqlsink",                     "sqlwritercleanupscript": "$$text.format('delete imp_fc.ship_ups_intdtl_tracking', windowstart, windowend)",                     "writebatchsize": 0,                     "writebatchtimeout": "00:00:00"                 },                 "translator": {                     "type": "tabulartranslator",                     "columnmappings": "company:company, upsd_ordn:upsd_ordn, upsd_bol:upsd_bol"                 }             },             "inputs": [                 {                     "name": "analyticsdb-shipups_03shp-01src_fc-915pm"                 }             ],             "outputs": [                 {                     "name": "analyticsdb-shipups_03shp-02dst_az-915pm"                 }             ],             "policy": {                 "timeout": "1.00:00:00",                 "concurrency": 1,                 "executionpriorityorder": "newestfirst",                 "style": "startofinterval",                 "retry": 3,                 "longretry": 0,                 "longretryinterval": "00:00:00"             },             "scheduler": {                 "frequency": "day",                 "interval": 1,                 "offset": "04:15:00"             },             "name": "915pm-shipups-fc-copy->[imp_fc]_[ship_ups_intdtl_tracking]"         },         {             "type": "copy",             "typeproperties": {                 "source": {                     "type": "sqlsource",                     "sqlreaderquery": "$$text.format('select distinct ups.upsd_bol, ups.upsd_bol imp_fc.ship_ups_intdtl_tracking ups left join olcm.ship_tracking st on ups.upsd_bol = st.ship_sys_track_num st.ship_sys_track_num null', windowstart, windowend)"                 },                 "sink": {                     "type": "sqlsink",                     "writebatchsize": 0,                     "writebatchtimeout": "00:00:00"                 },                 "translator": {                     "type": "tabulartranslator",                     "columnmappings": "upsd_bol:ship_sys_track_num, upsd_bol:ship_track_num"                 }             },             "inputs": [                 {                     "name": "analyticsdb-shipups_06shp-01src_az-915pm"                 }             ],             "outputs": [                 {                     "name": "analyticsdb-shipups_06shp-02dst_az-915pm"                 }             ],             "policy": {                 "timeout": "1.00:00:00",                 "concurrency": 1,                 "executionpriorityorder": "newestfirst",                 "style": "startofinterval",                 "retry": 3,                 "longretryinterval": "00:00:00"             },             "scheduler": {                 "frequency": "day",                 "interval": 1,                 "offset": "04:15:00"             },             "name": "915pm-shipups-az-update->[olcm]_[ship_tracking]"         }     ],     "start": "2017-08-22t03:00:00z",     "end": "2099-12-31t08:00:00z",     "ispaused": false,     "hubname": "adf-tm-prod-01_hub",     "pipelinemode": "scheduled" } }






Microsoft Azure  >  Azure Data Factory



Comments

Post a Comment

Popular posts from this blog

Azure DocumentDB Owner resource does not exist

having same error microsoft.azure.documents.documentclientexception: message: {"errors":["owner resource not exist"]} , scenario. when deployed webapp azure , try document docdb throws error. docdb exists in azure , contains document looking for. weird local machine(running vs) works fine. i'm using same settings in azure , local. have , idea this. thanks Microsoft Azure  >  Azure Cosmos DB
Read more

BizTalk Server 2013 Azure VM Log Shipping and HA for hosts

hi , background: for our customer, having a  2 biztalk , 1 sql setup of environment. [biztalk server 2013 azure vms provisioned azure gallery, sql server 2012 azure vm ]. in current configuration, there no high availability or disaster recovery plan set. want implement dr , possibly ha configuration on these servers. i have few questions regarding same. please let me know thoughts on same. currently writing log files (log shipping files) mounted/attached drive same sql server. how move these log files across secondary site since shared storage not supported. in case of sql server crash, losing log files since reside on same server. please share options on how can move these periodically between datacenters. for hosts pop3, ftp need host clustering (which requires biztalk machines present on windows cluster traditionally), please let know how achieve in azure scenario thanks , regards, ujjwal devarapalli -ujjwal you can refer article, issue azure vm lo...
Read more

How to send non-standard Content-Type header ?

hello, my custom client developed send custom content-type header "xyz" (with out "/"). message gone in lcs 2005. after have upgrade lcs 2005 ocs 2007 r2 in test environment. custom client not able send same custom content-type in ocs 2007 r2 used possible send in lcs 2005. below cut ocs 2007 r2 accessedge log file. tl_warn(tf_component) [0]0f68.0bb4::11/08/2010-06:02:51.546.00000de7 (sipstack,ccontenttypeheader::parse:contenttypeheader.cpp(118))( 00000000036249f0 ) exit - media-type syntax error in getting "/" tl_error(tf_component) [0]0f68.0bb4::11/08/2010-06:02:51.546.00000de8 (sipstack,csipmessage::parseallheaders:sipmessage.cpp(6408))( 0000000003566a50 ) exit. failed while parsing header. tl_error(tf_component) [0]0f68.0bb4::11/08/2010-06:02:51.546.00000de9 (sipstack,csiprequest::validateinboundheaders:siprequest.cpp(1391))( 0000000003566a50 ) exit - failed while parsing header. returned 0xc3e93f51(sipproxy_e_invalid_content_type_header)...
Read more