HDInsight Spark Streaming vs Stream Analytics

im looking scenario's use cases in 1 better suited vs other.

from initial research

a functional capabilities & learning curve & dev efforts

a. stream analytics quick going with, might inflexible in terms of nature of processing executed against stream.

e.g. if want mere aggregations (across records) or filtering (identify categories of records) or alerting (identify specific records) capabilities, stream analytics might effective option.

b. im not sure of leveraging reference data during stream processing supported in stream analytics. expect possible/available in spark streaming

e.g. event/record enrichment. i.e. read input stream event, used specific attributes, lookup additional attributes relevant event, , add stream event downstream processing.

or running complex co-relations across streams

e.g. event seen above threshold on 1 stream in last day, , relate-able event observed on stream, , points conclusion.

c..  , on.

b architecture & performance

im expecting able support same , maximum level of parallel processing on stream either on stream analytics or spark streaming. i.e. on count 2 options more or less similar in capabilities.

any advise, suggestions or references appreciated.

thanks you

alwyn 

 

hi,

disclaimer: i'm azure stream analytics (asa) team answer may little biased. 

thanks taking time write question. published blog post comparing azure stream analytics , spark here.

the first big difference want mention asa offers job service (i.e. serverless service), while spark has cluster form factor. gives advantages asa in term of ease of deployment , maintenance, , makes cost-efficient.

regarding questions: azure stream analytics has native support reference data, can augment streaming data. can use various inputs each job , corralate events between these inputs using join function. if have specific queries in mind, happy help.

let know if have further questions,

thanks,

js

Microsoft Azure  >  Azure Stream Analytics