Modern computer vision requires constant updates in technologies. Deployment of neural networks that can recognise and learn visual concepts without human supervision is the cutting-edge technology in this sphere. Latest innovation is deployment of GPT-4 in CV anomaly detection. Our expert, Kirill Starkov, Senior Machine Learning Engineer, will tell us about the newest research and will share his experience.
What models are used in CV?
Modern CV requires minimal human supervision and a high level of efficiency and accuracy. Development of special models trained to fulfil a limited range of tasks is a conventional way with a lot of additional expenses. To reduce costs, ML engineers try to reach a particular level of universality for neural networks. ‘Latest universal models based on neural networks are CLIP and GPT-4. Their deployment in CV is possible because they can be instructed in natural language for various tasks thanks to prompts and visual embedding’.
Also check: neural networks software
CLIP vs GPT-4
CLIP is a unique neural network, because it was trained on all information from the Internet. It is a multimodal model because it represents the notion in different varieties: as a picture and as a text (description). ‘CLIP can perform a great variety of classification benchmarks without direct optimisation for the benchmark’s performance. In CV, it is very useful, because we can determine the detection object without developing a separate detector’. We can say that the CLIP model is rather scalable and universal: it can be easily trained for different detection purposes with prompts and visual embedding and that saves money, calculation costs and time of clients. ‘My case of CLIP deployment in CV showed a 15% increase in accuracy’.
GPT-4 is a cutting-edge solution in neural networks. Like CLIP, it can process both text and images without supervision and long pre-training. ‘Compared to CLIP, GPT-4 works slower and requires fine-tuning. This fact can be considered as a disadvantage, but I believe that it increases accuracy and makes GPT-4 suitable for even more local tasks with rare conditions. Moreover, OpenAI is continuing development of this model: I’m sure we’ll see more powerful and precise solutions in the near future’.
Conclusions
Both CLIP and GPT-4 models show great performance. Deployment of both products showed that universality of multimodal neural networks is a cost-effective solution for any possible CV tasks. Of course, both models are not sufficiently stable, so there is always room for improvement.
