Putting these concepts into practice
After 4 years as the Watson Core Tooling lead architect building self-service tooling for the Watson Question Answering system, I joined the Developer Advocacy team of the Watson Data Platform organization which has the expanded mission of creating a platform that brings the portfolio of data and cognitive services to the IBM public cloud. Our mission was rather simple: win the hearts and minds of developers and help them be successful with their data and AI projects.
The work had multiple dimensions: education, evangelism, and activism. The first two are pretty straightforward, but the concept of activism is relevant to this discussion and worth explaining in more details. As the name implies, activism is about bringing change where change is needed. For our team of 15 developer advocates, this meant walking in the shoes of developers as they try to work with data—whether they're only getting started or already operationalizing advanced algorithms—feel their pain and identify the gaps that should be addressed. To that end, we built and made open source numerous sample data pipelines with real-life use cases.
At a minimum, each of these projects needed to satisfy three requirements:
- The raw data used as input must be publicly available
- Provide clear instructions for deploying the data pipeline on the cloud in a reasonable amount of time
- Developers should be able to use the project as a starting point for similar scenarios, that is, the code must be highly customizable and reusable
The experience and insights we gained from these exercises were invaluable:
- Understanding which data science tools are best suited for each task
- Best practice frameworks and languages
- Best practice architectures for deploying and operationalizing analytics
The metrics that guided our choices were multiple: accuracy, scalability, code reusability, but most importantly, improved collaboration between data scientists and developers.
