hadoop - spring XD rabbit source module fails to process messages, first message stays unacknowledged -

i trying simple spring xd application load log events in hdfs. have configured target application spring-ampq/rabbit log4j appender (the org.springframework.amqp.rabbit.log4j.amqpappender class) pump log messages pre-configured exchange. set following stream pull messages , push them hdfs, both soruce , sink modules off-the-shelf xd modules,

stream definition,

xd:>stream create --name demoq1 --definition "rabbit | hdfs --rollover=15 --directory=/user/root" --deploy 

created , deployed new stream 'demoq1'

xd:>stream list   stream name  stream definition                                   status   -----------  --------------------------------------------------  --------   demoq1       rabbit | hdfs --rollover=15 --directory=/user/root  deployed 

amqp appender publishing messages exchange , routing demoq1 queue, rabbit source picking first message , gets stuck, not acknowledge message. reason?

in container log, see this: "failed write message payload hdfs" ?

if so, need use type conversion between modules. rabbit source hdfs sink messages byte arrays.

your stream definition be,

stream create --name demoq1 --definition "rabbit --outputtype=text/plain | hdfs --rollover=15 --directory=/user/root" --deploy

or,

stream create --name demoq1 --definition "rabbit | hdfs --inputtype=text/plain --rollover=15 --directory=/user/root" --deploy

note outputtype or inputtype option in source/sink respectively. in case, hdfs sink's hdfsstoremessagehandler expects payload of type string.

for more details on type conversion, please check out: https://github.com/spring-projects/spring-xd/wiki/type-conversion