You're reading from The Machine Learning Solutions Architect Handbook Practical strategies and best practices on the ML lifecycle, system design, MLOps, and generative AI

Product type Paperback

Published in Apr 2024

Publisher Packt

ISBN-13 9781805122500

Length 602 pages

Edition 2nd Edition

Languages

Python

Tools

MLOps

Concepts

Machine Learning

Author (1):

David Ping

View More author details

Table of Contents (19) Chapters

Preface

1. Navigating the ML Lifecycle with ML Solutions Architecture FREE CHAPTER

2. Exploring ML Business Use Cases

3. Exploring ML Algorithms

4. Data Management for ML

5. Exploring Open-Source ML Libraries

6. Kubernetes Container Orchestration Infrastructure Management

7. Open-Source ML Platforms

8. Building a Data Science Environment Using AWS ML Services

9. Designing an Enterprise ML Architecture with AWS ML Services

10. Advanced ML Engineering

11. Building ML Solutions with AWS AI Services

12. AI Risk Management

13. Bias, Explainability, Privacy, and Adversarial Attacks

14. Charting the Course of Your ML Journey

15. Navigating the Generative AI Project Lifecycle

16. Designing Generative AI Platforms and Solutions

17. Other Books You May Enjoy

18. Index

Data ingestion

The data ingestion component plays a crucial role in acquiring data from diverse sources, including structured, semi-structured, and unstructured formats, such as databases, knowledge graph, social media, file storage, and IoT devices. Its primary responsibility is to store this data persistently in various storage solutions like object data storage (e.g., Amazon S3), data warehouses, or other data stores. Effective data ingestion patterns should incorporate both real-time streaming and batch ingestion mechanisms to cater to different types of data sources and ensure timely and efficient data acquisition.Various data ingestion technologies and tools cater to different ingestion patterns. For streaming data ingestion, popular choices include Apache Kafka, Apache Spark Streaming, and Amazon Kinesis/Kinesis Firehose. These tools enable real-time data ingestion and processing. On the other hand, for batch-oriented data ingestion, tools like Secure File Transfer Protocol (SFTP...