Nice article !

Actually, CharacterBERT can be very useful in many other domains, as Text to Code issues.

However, I think you miss an interessting comparison with BPE technique, which is also a technique to avoid vocabulary dependencies and use a character embeding for tokens.

--

--

--

Data scientist & Ph.D. researcher on AI. My area of expertise is around Deep Learning, NLP, and XAI — https://abdelkader-rhouati.medium.com/membership

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Abdelkader Rhouati

Abdelkader Rhouati

Data scientist & Ph.D. researcher on AI. My area of expertise is around Deep Learning, NLP, and XAI — https://abdelkader-rhouati.medium.com/membership

More from Medium

Spring 2022 College Admissions Roundup

The world of zebra crossing and digital currency tech /currency a thief a scammer a liars too…

10 Tips To Help You Find Success in a Career

The ultimate guide to Web3