Search icon CANCEL
Subscription
0
Cart icon
Cart
Close icon
You have no products in your basket yet
Save more on your purchases!
Savings automatically calculated. No voucher code required
Arrow left icon
All Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletters
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Talend Open Studio Cookbook

You're reading from  Talend Open Studio Cookbook

Product type Book
Published in Oct 2013
Publisher Packt
ISBN-13 9781782167266
Pages 270 pages
Edition 1st Edition
Languages
Author (1):
Rick Barton Rick Barton
Profile icon Rick Barton
Toc

Table of Contents (21) Chapters close

Talend Open Studio Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
1. Introduction and General Principles 2. Metadata and Schemas 3. Validating Data 4. Mapping Data 5. Using Java in Talend 6. Managing Context Variables 7. Working with Databases 8. Managing Files 9. Working with XML, Queues, and Web Services 10. Debugging, Logging, and Testing 11. Deploying and Scheduling Talend Code 12. Common Mistakes and Other Useful Hints and Tips Common Type Conversions
Management of Contexts Index

Processing multiple files at once


Often, with batch processes, it is required that multiple files are processed by the same job in a single tranche. This example shows how this can be achieved by merging a group of input files into a single output.

Getting ready

Open the jo_cook_ch08_0120_multipleFiles job. You will notice that it is currently reading a single file to a temporary file, and then copying the temporary fileto a permanent output.

How to do it...

The steps for processing multiple files at once are as follows:

  1. Add a tFileList component, open it, and set the directory to context.cookbookData+"/chapter8".

  2. Click on the + button under the Filemask box, and add the filemask "chapter08_jo_0120_customerData_*.txt".

  3. Your tFileList should look like the one shown, as follows:

  4. Move the OnSubjobOk from the tFileInputDelimited to the tFileList.

  5. Add a tJava component.

  6. Right-click on tFileList and select Row, then Iterate, and link to the tJava.

  7. Right-click on the tJava and select Trigger, then OnComponentOk...

lock icon The rest of the chapter is locked
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $15.99/month. Cancel anytime