Workflow, Tools, and Libraries in AI Projects

So far, we have discussed the background of big data, AI, AI problems, and methods. This chapter will discuss the workflow, tools, and libraries in AI projects, especially topics associated with security such as anomaly detection, spam detection, and so on. Technically speaking, nearly all AI projects have the same workflow, which includes the pre-processing (data collection, data cleaning, data annotation, etc.), the model training and testing/prediction, and fine-tuning (especially with the development of a transformer-based large model). However, there are various tools and libraries for different tasks. Therefore, we will discuss the tools and libraries within specific tasks.

All in all, in this chapter, the following topics are discussed:

• Workflow of AI projects
• Tools and libraries for visual network traffic analysis
• Tools and libraries for malware detection