Word Embeddings for Arabic (arabic_w2v_cc_300d)

Description

This model is trained on Common Crawl and Wikipedia using fastText. It is trained using CBOW with position-weights, in dimension 300, with character n-grams of length 5, a window of size 5 and 10 negatives.

The model gives 300 dimensional vector outputs per token. The output vectors map words into a meaningful space where the distance between the vectors is related to semantic similarity of words.

These embeddings can be used in multiple tasks like semantic word similarity, named entity recognition, sentiment analysis, and classification.

Live Demo Open in Colab Download Copy S3 URI

How to use

Use as part of a pipeline after tokenization.

...
embeddings = WordEmbeddingsModel.pretrained("arabic_w2v_cc_300d", "ar") \
.setInputCols(["document", "token"]) \
.setOutputCol("embeddings")
nlp_pipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings])
pipeline_model = nlp_pipeline.fit(spark.createDataFrame([[""]]).toDF("text"))
result = pipeline_model.transform(spark.createDataFrame([['أنا أحب التعلم الآلي']], ["text"]))

...
val embeddings = WordEmbeddingsModel.pretrained("arabic_w2v_cc_300d", "ar") 
.setInputCols(Array("document", "token"))
.setOutputCol("embeddings")
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings))
val data = Seq("أنا أحب التعلم الآلي").toDF("text")
val result = pipeline.fit(data).transform(data)

import nlu

text = ["أنا أحب التعلم الآلي"]
arabicvec_df = nlu.load('ar.embed.cbow.300d').predict(text, output_level='token')
arabicvec_df

Results

The model gives 300 dimensional Word2Vec feature vector outputs per token.

|                   ar_embed_cbow_300d_embeddings	token
|----------------------------------------------------|--------	
| [-0.11158058792352676, -0.06634224951267242, -...	أنا
| [-0.2818698585033417, -0.21061033010482788, -0...	أحب

Model Information

Data Source

This model is imported from https://fasttext.cc/docs/en/crawl-vectors.html