In part one of this blog series, we detailed what all is necessary in order to effectively train a machine learning model and discussed some best practices for training models. To conclude this blog series, this post will highlight challenges faced when building machine learning models and offer tips and tricks for overcoming those roadblocks.
Challenges When Building Machine Learning Models
In spite of the best practices highlighted in part one, building and training machine learning models does not come without challenges.
Take the resources you need to actually build models as one. You need a lot of computing resources, especially if you’re trying to build good models. For example, video datasets may take longer to process than image datasets. Simple datasets can be trained using only the CPU and should take only a couple of minutes. More complex deep learning models, on the other hand, require anywhere from 2 to 32 GPUs and days to train since these are generally intractable problems.
A lack of computing resources constrains the development process because training the models takes a significant amount of time. And if you’re looking to the cloud as a solution, that could be a costly answer because machine learning models run for a long time and consume a lot of GPU resources.
Another challenge involves understanding statistics and mathematical principles before collecting data and training models. In many cases, you won’t know the data is garbage if you do not understand statistical principles like selection bias, measurement bias, Simpson’s paradox, statistical significance, etc. It might be wise to have a resident data scientist to make sense of the data before machine learning engineers build models out of it. Data collection can be a very difficult, time-consuming process — you’re not just going to wake up one day with the training data you need. If you’re thinking of building a model today, then the first step is spending the next couple of months just collecting data and trying to clean it, ensuring it’s right for your model. And just because you have a bunch of data doesn’t mean it was collected in the right way and effectively prepared for your model.
A common mistake when building a machine learning model is expecting to simply punch the data in and get something of relevance out. If you don’t understand your model’s hyperparameters, it will be difficult to tune your model to the right form in order to actually pull out the best results. Without an effective understanding of statistics and mathematical principles, you’re limited in what you can do with your algorithms.
Finally, overfitting and underfitting data is one of the main causes of poor performance. Overfitting refers to a model that relies too heavily on the training data. When you try to use an overfit model with new, unseen data, the model won’t perform well because it has memorized the relationship in your data, and it’s very good at predicting the outcome given the data that you’ve trained it on. On the flip side of that, underfitting is when your model can neither accurately model the training data nor new data because it was taught using too little training data. In a perfect world, you want to select a model that’s neither underfit nor overfit but a good fit.
To find that ideal fit, think about training your model with 70% of the collected data and use the remaining 30% to test whether your model actually works, a technique known as cross-validation. How you split your datasets into training and testing samples may vary depending on the model you are building, however, the principle is the same. If the model works, the chance it will also work on unseen, real-world data is great. But if not, you still have the option to tune your model to make sure it works with the data you’ve reserved for testing.
One common misconception is that machine learning can solve every problem. But unfortunately, there’s no general applicability to every sort of problem — machine learning is problem-specific, so you need to fully understand the problem you’re trying to solve and have the data to support it before you even think of using machine learning. Remember to keep the resources you need to build the model in mind, collect a sufficient amount of the right data and comprehend the statistics and mathematical principles behind your model to overcome the most common challenges in building machine learning models.
Today, machine learning is more widely available and easier to access — even for nontechnical folks — than it’s ever been thanks to APIs provided by tech companies like Amazon. But that availability and ease of access means people are likely to use machine learning without fully understanding it, which could lead to trouble if they start making business decisions based on their results.
At Cardinal Peak, we uniquely understand machine learning and have employed it across a number of projects, from edge machine learning for remote video, Bluetooth headphones with active noise cancellation and mobile apps to video streaming and medical test diagnosis. If you’re looking to leverage machine learning for your next project, let us know how we can help!
Additional Resources
For more information, check out our other blog posts on training machine learning models and using Amazon SageMaker for sentiment analysis: