Architecting the PsyStock ML platform
There is a set of desirable tenets that we can define for our ML platform based on a distillation of the research on best practices and example reference architectures. The main tenets that we want to maintain in our platform are the following:
- Leverage open systems and standards: Using open systems such as the ones available in MLflow allows longevity and flexibility to leverage the open source community advances and power to extend the company ML platform at a lower cost.
- Favor scalable solutions: A company needs to be prepared for a future surge in growth; although this is the first version, the ability to surge on-demand from training and perspective needs to be in place.
- Integrated reliable data life cycle: Data is the center of gravity of the ML platform and should be managed in a reliable and traceable manner at scale.
- Follow SWE best practices: For example, separation of concerns, testability, CI/CD, observability, and...
