Problems encountered while using a deep learning model for Computer Vision
Introduction
An ML Engineer never really codes a deep learning algorithm from scracth. Instead they spend a lot of time in gathering the data and cleaning the data. Once the data is cleaned, they try to find an open source library or an open source github code and integrate the specific algorithm into their module and run experiments to evaluate the specific model. During this process I have noticed a lot of problems and in this post I am addressing the problems.
Problems encountered
- Setting up Environments : Each Machine Learning/ Deep Learning algorithm need specific environments and it takes a lot of time to setup environments. Sometimes it goes to an extent of setting environments with specific versions to run a specific code.
- Preprocessing of Data : Different frameworks need different formats to feed the data into the model. For example Tensorflow has a specific way of preprocessing data and that is very diferent to how pytorch needs the data to be fed in. It plays a very important role.
- Training the Model: Each model has a specific way of how the model is trained and how hyperparameters are used to train and fine tune the model.
- Saving/Loading Model : Saving the model is in different formats and loading the model is in different strucutre.
Few Examples :
Here we see that , preprocessing technique is very different in both frameworks.
Now, the bigger question. How can we solve this problem ?
One possible solution is to see commonalities between the frameworks i.e if both frameworks need input as a string which only requires a path then its better to build a generic input function and a generic output function and just change the algorithm with the frame work. This helps in saving time and focusing more on running experiments and working on the metrics.
Resources:
