Google AI team recently open-source BiT (Big Transfer) for general visual representation learning. Current computer vision training generally involves a pre-trained model due to lack of labeled data for computer vision tasks. This has been a common problem for computer vision scientists to collect and train models with a large set of generic data available via tools like OpenImages or Places, but many times this collection of a large set of data (over 1M labeled images) could be prohibitive for an average practitioner.
The available solutions include using pre-trained models that are trained on generic data (example: ImageNet). Despite these pre-trained models working well in practice, this still doesn’t solve the problem for conditions like grasping news concepts and then to understand them in a different context. Just like how BERT and T5 have shown advances in the language domain, it is believed BiT based large-scale pre-training can advance the performance of computer vision models.
Paper: https://arxiv.org/abs/1912.11370
Github: https://github.com/google-research/big_transfer
BiT Models – Components of all TL blocks: https://tfhub.dev/google/collections/bit/1
Asif Razzaq is an AI Journalist and Cofounder of Marktechpost, LLC. He is a visionary, entrepreneur and engineer who aspires to use the power of Artificial Intelligence for good.
Asif's latest venture is the development of an Artificial Intelligence Media Platform (Marktechpost) that will revolutionize how people can find relevant news related to Artificial Intelligence, Data Science and Machine Learning.
Asif was featured by Onalytica in it’s ‘Who’s Who in AI? (Influential Voices & Brands)’ as one of the 'Influential Journalists in AI' (https://onalytica.com/wp-content/uploads/2021/09/Whos-Who-In-AI.pdf). His interview was also featured by Onalytica (https://onalytica.com/blog/posts/interview-with-asif-razzaq/).
