Machine Learning-based System Development Process Model: Difference between revisions
Jump to navigation
Jump to search
m (Text replacement - ". " to ". ") |
m (Text replacement - "<P> [[" to "<P> [[") |
||
| Line 35: | Line 35: | ||
=== 2016 === | === 2016 === | ||
* http://databricks.com/blog/2016/05/31/apache-spark-2-0-preview-machine-learning-model-persistence.html | * http://databricks.com/blog/2016/05/31/apache-spark-2-0-preview-machine-learning-model-persistence.html | ||
** QUOTE: So far, we have only looked at saving and loading a single ML model. In practice, [[ML workflow]]s consist of many stages, from feature extraction and transformation to model fitting and tuning. [[MLlib]] provides [[MLlib Pipeline|Pipeline]]s to help users construct these workflows. (See this notebook for a tutorial on [[ML Pipeline]]s analyzing a bike sharing dataset.) <P> | ** QUOTE: So far, we have only looked at saving and loading a single ML model. In practice, [[ML workflow]]s consist of many stages, from feature extraction and transformation to model fitting and tuning. [[MLlib]] provides [[MLlib Pipeline|Pipeline]]s to help users construct these workflows. (See this notebook for a tutorial on [[ML Pipeline]]s analyzing a bike sharing dataset.) <P> [[MLlib]] allows users to save and load entire Pipelines. Let’s look at how this is done on an example Pipeline with these steps: | ||
*** Feature extraction: Binarizer to convert images to black and white | *** Feature extraction: Binarizer to convert images to black and white | ||
*** Model fitting: Random Forest Classifier to take images and predict digits 0–9 | *** Model fitting: Random Forest Classifier to take images and predict digits 0–9 | ||
Latest revision as of 01:47, 27 February 2024
A Machine Learning-based System Development Process Model is a software development process model to develop machine learning-based systems.
- Context:
- It can (typically) be referenced by a Machine Learning Development Team.
- It can (typically) include ML Systems Analysis, ML Software Design, ML Software Development, ML Software Testing, ML Software Documentation, SML oftware Evaluation, an ML Software Maintenance.
- It can be associated to a Machine Learning Development Methodology.
- It can be associated to a Data Science Workflow.
- It can be supported by a Machine Learning Workflow System, such as Spark MLlib Pipeline, TensorFlow ML Workflow, Uber's Michelangelo, and Kubeflow.
- It can (often) produce an ML Model Training System that creates trained ML models.
- It can (often) contain Model Data Preparation Code.
- …
- Example(s)
- Counter-Example(s):
- See: Business Process Model, ML Software Design, ML Software Development, ML Software Maintenance, Production Data Store, ML Pipeline.
References
2019b
- https://www.kubeflow.org/
- QUOTE: ... The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable.
2017
- https://developers.google.com/machine-learning/guides/text-classification/
- QUOTE: Here’s a high-level overview of the workflow used to solve machine learning problems:
- QUOTE: Here’s a high-level overview of the workflow used to solve machine learning problems:
2016
- http://databricks.com/blog/2016/05/31/apache-spark-2-0-preview-machine-learning-model-persistence.html
- QUOTE: So far, we have only looked at saving and loading a single ML model. In practice, ML workflows consist of many stages, from feature extraction and transformation to model fitting and tuning. MLlib provides Pipelines to help users construct these workflows. (See this notebook for a tutorial on ML Pipelines analyzing a bike sharing dataset.)
MLlib allows users to save and load entire Pipelines. Let’s look at how this is done on an example Pipeline with these steps:
- Feature extraction: Binarizer to convert images to black and white
- Model fitting: Random Forest Classifier to take images and predict digits 0–9
- Tuning: Cross-Validation to tune the depth of the trees in the forest
- QUOTE: So far, we have only looked at saving and loading a single ML model. In practice, ML workflows consist of many stages, from feature extraction and transformation to model fitting and tuning. MLlib provides Pipelines to help users construct these workflows. (See this notebook for a tutorial on ML Pipelines analyzing a bike sharing dataset.)