MIS 655 Topic 7 DQ 2
docx
keyboard_arrow_up
School
Grand Canyon University *
*We aren’t endorsed by this school
Course
MIS 655
Subject
Information Systems
Date
Feb 20, 2024
Type
docx
Pages
2
Uploaded by MasterTitanium11775
You previously built a model that was designed to predict when customers would default. It
included macro-economic variables such as unemployment rate, GDP, and other market
indicators. When the model was initially built, it proved to be quite accurate. One year later,
the model's accuracy has decreased significantly. What could have occurred during that time
to reduce the efficacy of the model? When building models, should one expect them to work
indefinitely? Why? What could be done to keep the model current? Provide two or three
examples.
We can expect that the previously built model won’t properly work forever because of
model staleness or data drift.
ML projects are dynamic systems highly dependent on input data. In contrast to
conventional software, most of them degrade over time or become more and more
irrelevant. This problem is also known as model staleness. The following instances
could occur over time which may reduce the efficacy of the model:
Data drift – when the distribution of input feature i.e., independent variables,
changes drastically from what the model has seen in training.
Model or concept drift – when the properties of target variables i.e. dependent
variables, change without changing the input features.
Training-serving skew – model in production, doesn’t have the same
performance as training.
Technical bugs and other similar things.
In addition to monitoring, an antidote to model staleness is the implemented
retraining strategy. The time of model retraining depends on the business use case,
but in general, there are four different approaches:
Based on time interval – retrain the model every day, week, month, or similar.
Performance-based – retrain the model when the performance of the model
goes under a predefined threshold.
Based on data changes – trigger training after significant data shifts or after
introducing new features.
Retrain on demand – manually retrain the model for some other reason.
When designing a machine learning system it is important to understand how your
data is going to change over time. A well-architected system should take this into
account, and a plan should be put in place for keeping your models updated.
Manual retraining - One way to maintain models with fresh data is to train and
deploy the models using the same process used to build the models in the first
place. As we are manually retraining the models, we may discover a new
algorithm or a different set of features that provide improved accuracy.
Continuous learning - Another way to keep your models up-to-date is to have an
automated system to continuously evaluate and retrain your models. This type
of system is often referred to as continuous learning.
Companies like Spotify and Netflix use ML models to recommend music or TV shows
based on the user's previous listening and viewing history. Over time and with
training, these algorithms aim to understand user's preferences to accurately predict
which artists or films you may enjoy. Preferences may change over time so these
models
need
to
be
in
trained
continuously.
Another example would be the stock market performance prediction based on year-to-
year analysis. To maintain data accuracy and model effectiveness, these models need
to continuously learn from new data that is latest movements in the market.
Google uses machine learning to build models of how long trips will take based on
historical traffic data (gleaned from satellites). It then takes that data based on your
current trip and traffic levels to predict the best route according to these factors,
continuous learning allows instant adaptation to new input for providing relevant,
high-quality information.
Caruana, V. (2023). Algolia.
How continuous learning lets machine learning provide
increasingly accurate predictions and recommendations
https://www.algolia.com/blog/ai/how-continuous-learning-lets-machine-learning-
provide-increasingly-accurate-predictions-and-recommendations/
Zvornicanin, E. (2023). Neptune.ai.
Deploying ML Models
https://neptune.ai/blog/deploying-ml-models-make-sure-new-model-is-better
Your preview ends here
Eager to read complete document? Join bartleby learn and gain access to the full version
- Access to all documents
- Unlimited textbook solutions
- 24/7 expert homework help