BioBERT Embeddings (Pubmed)

Description

This model contains a pre-trained weights of BioBERT, a language representation model for biomedical domain, especially designed for biomedical text mining tasks such as biomedical named entity recognition, relation extraction, question answering, etc. The details are described in the paper “BioBERT: a pre-trained biomedical language representation model for biomedical text mining”.

Live Demo Open in Colab Download Copy S3 URI

How to use

embeddings = BertEmbeddings.pretrained("biobert_pubmed_base_cased", "en") \
.setInputCols("sentence", "token") \
.setOutputCol("embeddings")
nlp_pipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings])
pipeline_model = nlp_pipeline.fit(spark.createDataFrame([[""]]).toDF("text"))
result = pipeline_model.transform(spark.createDataFrame([['I hate cancer']], ["text"]))

val embeddings = BertEmbeddings.pretrained("biobert_pubmed_base_cased", "en")
.setInputCols("sentence", "token")
.setOutputCol("embeddings")
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings))
val data = Seq("I hate cancer").toDF("text")
val result = pipeline.fit(data).transform(data)

import nlu

text = ["I hate cancer"]
embeddings_df = nlu.load('en.embed.biobert.pubmed_base_cased').predict(text, output_level='token')
embeddings_df

Results

token	en_embed_biobert_pubmed_base_cased_embeddings

	I	[0.4227580428123474, -0.01985771767795086, -0....
	hate	[0.04862901195883751, 0.2535072863101959, -0.5...
	cancer	[0.05491625890135765, 0.09395376592874527, -0....

Model Information

Data Source

The model is imported from https://github.com/dmis-lab/biobert