Global AI and Data Science

Global AI & Data Science

Train, tune and distribute models with generative AI and machine learning capabilities

 View Only

  • 1.  RoBERTa: a Robustly Optimized BERT Pretraining Approach

    Posted Wed September 04, 2019 12:55 AM
    Edited by System Admin Fri January 20, 2023 04:09 PM
    How do you optimize language model pre-training when training tends to be computationally expensive and executed on differing datasets? Maybe RoBERTa has the answers. Facebook's pre-training recipe appears to have greatly improved on BERT's bench-marking performance. What do you think is in store for RoBERTa?
    the amount of mined parallel sentences for most of the language pairsImage from: https://github.com/facebookresearch/LASER/blob/master/tasks/WikiMatrix/WikiMatrix-sizes.pdf


    ------------------------------
    William Roberts
    ------------------------------
    #GlobalAIandDataScience
    #GlobalDataScience


  • 2.  RE: RoBERTa: a Robustly Optimized BERT Pretraining Approach

    Posted Mon September 23, 2019 06:31 PM
    RoBERTa is surely going to drop out of SOTA, soon!

    ------------------------------
    William Roberts
    ------------------------------

    Original Message


Related Content