Global AI and Data Science

Global AI & Data Science

Train, tune and distribute models with generative AI and machine learning capabilities


#Artificialintelligence
#Datascience
 View Only

  • 1.  RoBERTa: a Robustly Optimized BERT Pretraining Approach

    Posted 09/04/19 12:55 AM
    Edited by System Admin 01/20/23 04:09 PM
    How do you optimize language model pre-training when training tends to be computationally expensive and executed on differing datasets? Maybe RoBERTa has the answers. Facebook's pre-training recipe appears to have greatly improved on BERT's bench-marking performance. What do you think is in store for RoBERTa?
    the amount of mined parallel sentences for most of the language pairsImage from: https://github.com/facebookresearch/LASER/blob/master/tasks/WikiMatrix/WikiMatrix-sizes.pdf


    ------------------------------
    William Roberts
    ------------------------------
    #GlobalAIandDataScience
    #GlobalDataScience


  • 2.  RE: RoBERTa: a Robustly Optimized BERT Pretraining Approach

    Posted 09/23/19 06:31 PM
    RoBERTa is surely going to drop out of SOTA, soon!

    ------------------------------
    William Roberts
    ------------------------------



Related Content