In-House Machine Learning Operations (MLOps) Platform

An In-House Machine Learning Operations (MLOps) Platform is an organization-internal machine learning operations data processing platform system that supports in-house ML system development tasks.

• AKA: In-House ML Platform, In-House ML Ops Platform, Internal Machine Learning Ops Platform System, In-House Machine Learning Platform.
• Context:
  • It can perform In-House ML Model Management through in-house ML model registry systems.
  • It can enable In-House ML Model Training through in-house ML training infrastructures.
  • It can support In-House ML Pipeline Orchestration through in-house ML workflow management systems.
  • It can facilitate In-House ML Feature Engineering through in-house ML feature store platforms.
  • It can provide In-House ML Model Deployment through in-house ML serving infrastructures.
  • ...
  • It can integrate In-House ML Platform Components with in-house data infrastructures.
  • It can connect In-House ML Development Environments with in-house ML production environments.
  • It can orchestrate In-House ML Workflows with in-house ML scheduling systems.
  • It can monitor In-House ML Model Performance with in-house ML monitoring tools.
  • ...
  • It can range from being a Simple In-House Machine Learning Operations Platform to being a Complex In-House Machine Learning Operations Platform, depending on its in-house ML platform capability scope.
  • It can range from being a Cloud-Based In-House Machine Learning Operations Platform to being an On-Premises In-House Machine Learning Operations Platform, depending on its in-house ML platform deployment model.
  • It can range from being a Custom In-House Machine Learning Operations Platform to being a Hybrid In-House Machine Learning Operations Platform, depending on its in-house ML platform third-party integration level.
  • ...
  • It can be managed by In-House ML Platform Operations Teams for in-house ML platform maintenance tasks.
  • It can be utilized by In-House ML Engineers for in-house ML engineering tasks.
  • It can contain In-House ML Platform System Components for in-house ML platform functionalitys.
  • It can implement In-House ML Platform Security Protocols for in-house ML data protections.
  • It can enforce In-House ML Platform Governance Policys for in-house ML compliance requirements.
  • ...
• Example(s):
  • Custom In-House Machine Learning Operations Platforms, such as:
    • Netflix In-House Machine Learning Operations Platform (including Netflix Metaflow).
    • Uber In-House Machine Learning Operations Platform (Uber Michelangelo).
    • Meta In-House Machine Learning Operations Platform (Meta FBLearner).
    • Airbnb In-House Machine Learning Operations Platform.
    • PlayStation In-House Machine Learning Operations Platform.
    • Salesforce In-House Machine Learning Operations Platform (Salesforce Einstein Platform).
  • Hybrid In-House Machine Learning Operations Platforms, such as:
    • Medable In-House Machine Learning Operations Platform (2022), based on Google Cloud VertexAI.
    • SIE In-House Machine Learning Operations Platform (SIE Mastermind) (2022), based on Databricks ML Platform.
    • OpenGov In-House Machine Learning Operations Platform (2016), implementing Apache Spark ML.
    • AT&T Wireless In-House Machine Learning Operations Platform (2005), implementing SAS ML Platform.
  • Specialized In-House Machine Learning Operations Platforms, such as:
    • Viglink In-House Machine Learning Operations Platform (2014), for in-house ML recommendation systems.
    • Weights & Biases In-House Machine Learning Operations Platform, for in-house ML experiment trackings.
  • ...
• Counter-Example(s):
  • ETL Platform, which lacks in-house ML model lifecycle management.
  • Software Development Operations System, which lacks in-house ML-specific tooling.
  • Product Analytics System, which lacks in-house ML model training capability.
  • Business Intelligence Platform, which lacks in-house ML pipeline orchestration.
  • Third-Party ML Platform, which lacks in-house customization control.
• See: Machine Learning Operations Platform, ML Platform Engineering, ML System Development, MLaaS Platform, Machine Learning Framework.

---

---

References

2023

• https://jobs.netflix.com/jobs/278437235
  • The Machine Learning Platform (MLP) provides the foundation for all of this innovation. It offers ML/AI practitioners across Netflix the means to achieve the highest possible impact with their work by making it easy to develop, deploy and improve their machine-learning models.

2018

• "Fact Store at Scale for Netflix Recommendations." Presentation at SparkSummit, 2018
  • QUOTE: ... As a data driven company, we use Machine Learning algos and A/B tests to drive all of the content recommendations for our members. To improve the quality of our personalized recommendations, we try an idea offline using historical data. Ideas that improve our offline metrics are then pushed as A/B tests which are measured through statistically significant improvements in core metrics such as member engagement, satisfaction, and retention. The heart of such offline analyses are historical facts data that are used to generate features required by the machine learning model. For example, viewing history of a member, videos in mylist etc. ...

---