diff --git a/.gitignore b/.gitignore index 10ad02ef32c99c292a43aaec5bd41314cb7508ee..2f0ed0ea4e99e44b86960625170d2b69e1686379 100644 --- a/.gitignore +++ b/.gitignore @@ -154,4 +154,6 @@ subset* time* -/data* \ No newline at end of file +/data* + +output_bert_allcooc_adjsampling3radius20km_batch32_epoch10 \ No newline at end of file diff --git a/README.md b/README.md index 37971368cc954b9404d08645a02e035c2dbd3ca6..9bd2998e60b556fbdefd92d5a4ab04e033855a70 100644 --- a/README.md +++ b/README.md @@ -93,3 +93,14 @@ grid.run() | -e,--epochs | number of epochs | | -d,--dimension | size of the ngram embeddings | | --admin_code_1 | (Optional) If you wish to train the network on a specific region | + + +# New model based on BERT embeddings + +In the recent years, BERT architecture proposed by Google researches enables to outperform state-of-art methods for differents tasks in NLP (POS, NER, Classification). To verify if BERT embeddings would permit to increase the performance of our approach, we code a script to use bert with our data. In our previous model, the model returned two values each on between [0,1]. Using Bert, the task has shifted to classification (softmax) where each class correspond to a cell on the glob. We use the hierarchical projection model : Healpix. Other projections model like S2geometry can be considered : https://s2geometry.io/about/overview. + +In order, to run this model training, run the `bert.py` script : + + python3 bert.py <train_dataset> <test_dataset> + +The train and test dataset are table data composed of two columns: sentence and label. \ No newline at end of file