Spring Batch – Read once, Write multi example

Sometimes, we need to read once, write multi ETL with Spring Batch. Spring Batch supports it with CompositeItemWriter (org.springframework.batch.item.support.CompositeItemWriter).

Sample table schema

CREATE TABLE "TB_SOURCE"
   (	"COL1" VARCHAR2(10 BYTE),
	"COL2" VARCHAR2(20 BYTE),
	"COL3" VARCHAR2(20 BYTE)
   );

CREATE TABLE "TB_TARGET1"
   (	"NEW_COL1" VARCHAR2(10 BYTE),
	"NEW_COL2" VARCHAR2(20 BYTE)
   );

  CREATE TABLE "TB_TARGET2"
   (	"NEW_COL1" VARCHAR2(10 BYTE),
	"NEW_COL3" VARCHAR2(20 BYTE)
   );

Declaration of CompositeItemWriter

<bean id="TestCompositeWriter"
class="org.springframework.batch.item.support.CompositeItemWriter"
scope="step">
<property name="delegates">
<list>
<ref bean="TestWriter1"/>
<ref bean="TestWriter2"/>
</list>
</property>
</bean>
<bean id="TestWriter1"
class="org.springframework.batch.item.database.JdbcBatchItemWriter"
scope="step">
<property name="assertUpdates" value="true" />
<property name="itemSqlParameterSourceProvider">
<bean class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
</property>
<property name="sql"
value="INSERT INTO TB_TARGET1(NEW_COL1, NEW_COL2)
VALUES(:newCol1, :newCol2)"
/>
<property name="dataSource" ref="testDataSource" />
</bean>
<bean id="TestWriter2"
class="org.springframework.batch.item.database.JdbcBatchItemWriter"
scope="step">
<property name="assertUpdates" value="true" />
<property name="itemSqlParameterSourceProvider">
<bean class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" />
</property>
<property name="sql"
value="INSERT INTO TB_TARGET2(NEW_COL1, NEW_COL3)
VALUES(:newCol1, :newCol3)"
/>
<property name="dataSource" ref="testDataSource" />
</bean>

Make sure that CompositeItemWriter has 2 delegates (TestWriter1, TestWriter2). You can use any Writer as it’s delegate.

Full Job declaration

<batch:job id="TestCompositeJob" job-repository="jobRepository">
<batch:step id="SimpleStep">
<batch:tasklet transaction-manager="testTransactionManager" allow-start-if-complete="true">
<batch:chunk
reader="TestReader"
processor="TestProcessor"
writer="TestCompositeWriter"
commit-interval="1"
reader-transactional-queue="false"
/>
</batch:tasklet>
</batch:step>
</batch:job>

Make sure that Job task is using new CompositeItemWriter instead of JdbcBatchItemWriter.

 

You can download the full sample source at https://github.com/tkstone/spring_batch_sample01

  • Spring context : src/main/resources/spring/job-test-composite-context.xml
  • Launcher class : test.main.TestCompositeRun.java
  • Processor class : test.main.TestProcessor.java

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.