Design and implementation of a generic MLOps Platform

In the Advanced Analytics team of the Research and Business Intelligence (OBI) department, there was a need to bring various use cases, each with its own focus, to production.

This required a generic MLOps platform on which all use cases can be centrally brought to production. The platform must meet multiple security aspects in accordance with the municipality’s policy: network isolation, data privacy, separated rights with regard to the principle of minimal privileges, automated CI/CD street and robust product lifecycle management. In addition, it was also necessary that all accumulated knowledge and experience, in the respective professional groups, is centrally secured. This was necessary for scaling up, automating and standardizing the AI/ML services provided.

Contribution

CIMSOLUTIONS came on board with expertise in various roles including Machine Learning Engineer, Data Scientist, AI Architect, Data Platform Engineer, DevOps Engineer, Data Engineer, Cloud Engineer, Cloud Network Specialist, Software Engineer.

CIMSOLUTIONS refined the initial setup of the MLOps platform on Azure Cloud based on the needs of the municipality and then implemented it in the form of IaC (Infrastructure-as-Code), which ensures the systematic rollout of comparable environments (OTAP street). CIMSOLUTIONS also developed the solution patterns of the MLOps platform (batch, real-time, scheduled model deployments) and applied them to the specific business cases. Together with architects and product owners of the municipality, CIMSOLUTIONS played a key role in establishing and documenting the working agreements around Identity and Access Management (IAM), Data and Model Management, Cost Management (FinOps), Platform Reliability and Monitoring. CIMSOLUTIONS regularly provides up-to-date knowledge about best practices in the field of data modeling, software engineering, DevOps and MLOps to internal specialists of the Municipality.

Methods and Techniques

The team used advanced technologies such as MS Azure Cloud, MS Azure Machine Learning, MS Azure DevOps, Azure Synapse, Azure Databricks, MS Fabric, ADLS Gen 2, Azure KeyVault, Azure Artifacts, Azure App Service, Azure Virtual Network, Private Endpoints, Hub-Spoke network topology, Azure Firewall, Network Security Groups, Custom DNS, Azure ML CLI 2.55+, Python 3.8+, Azure ML Python SDK V1.48+, MLflow 2.1.1+, TensorFlow 2.4+, Pytorch 1.7+, Yolo v5+, LangChain 0.2.3+, Streamlit 1.35+, in an Agile/Scrum environment to build the state-of-the-art platform.

Results

A Generic and Scalable MLOps Platform on which all use cases can be centrally brought to production.
A Secure Infrastructure that complies with the municipality’s security policy, including network isolation, data privacy, separate rights with regard to the principle of minimum privileges.
Established Working Agreements regarding Identity and Access Management (IAM), Data and Model Management, Cost Management (FinOps), Platform Reliability and Monitoring.
Regular Knowledge Transfer and Knowledge Retention of up-to-date knowledge on best practices in the latest AI/ML technologies and DevOps practices.