BERT Base Biolink Embeddings

Description

This embeddings component was trained on PubMed abstracts all along with citations link information. The embeddings were introduced in this paper. This model achieves state-of-the-art performance on several biomedical NLP benchmarks such as BLURB and MedQA-USMLE.

Predicted Entities

Live Demo Open in Colab Download Copy S3 URI

How to use

embeddings = BertEmbeddings.pretrained("bert_biolink_base", "en")\
.setInputCols(["sentence", "token"])\
.setOutputCol("embeddings")

val embeddings = BertEmbeddings.pretrained("bert_biolink_base", "en")
.setInputCols(Array("sentence", "token"))
.setOutputCol("embeddings")

import nlu
nlu.load("en.embed.e").predict("""Put your text here.""")

Model Information

References

https://pubmed.ncbi.nlm.nih.gov/

@InProceedings{yasunaga2022linkbert,
author =  {Michihiro Yasunaga and Jure Leskovec and Percy Liang},
title =   {LinkBERT: Pretraining Language Models with Document Links},
year =    {2022},  
booktitle = {Association for Computational Linguistics (ACL)},  
}

Benchmarking

Scores for several benchmark datasets :

- BLURB : 83.39
- PubMedQA : 70.2
- BioASQ : 91.4
- MedQA-USMLE : 40.0