Training Computer Vision Transformers without Natural Images

Can Vision Transformers Learn without Natural Images?

Deep AI https://deepai.org/publication/can-vision-transformers-learn-without-natural-images

They pre-train Vision Transformers without any image collections and annotation labor. They experimentally verify it partially outperforms Self-Supervised Learning without using any natural images in the pre-training phase. It can interpret natural image datasets to a large extent. For example, the performance rates on the CIFAR-10 dataset are as follows: their proposal 97.6% vs. SimCLRv2 97.4% vs. ImageNet 98.0%

arXiv paper abstract https://arxiv.org/abs/2103.13023

arXiv PDF paper https://arxiv.org/pdf/2103.13023v1.pdf

Unlimited computer fractals can help train AI to see

Large datasets like ImageNet have supercharged the last 10 years of AI vision, but they are hard to produce and contain bias. Computer generated datasets provide an alternative.

M.I.T. Technology Review https://www.technologyreview.com/2021/02/04/1017486

Web site https://hirokatsukataoka16.github.io/Vision-Transformers-without-Natural-Images

GitHub https://github.com/hirokatsukataoka16/FractalDB-Pretrained-ResNet-PyTorch

Oral presentation http://hirokatsukataoka.net/pdf/accv20_kataoka_oral.pdf

Poster http://hirokatsukataoka.net/pdf/accv20_kataoka_poster.pdf

Please like and share this post if you enjoyed it using the buttons at the bottom!

Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact

Web site with my other posts by category https://morrislee1234.wixsite.com/website

#ImageNet #Transformers #ComputerVision #Fractal #AINewsClips #AI #ML #ArtificialIntelligence #MachineLearning