Add zenodo link

a2fe7801 · George Marchment · bf632200 · a2fe7801 · a2fe7801
Commit a2fe7801 authored 1 year ago by George Marchment
--- a/README.md
+++ b/README.md
 # Github-Crawler
+[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-green.svg)](https://www.gnu.org/licenses/gpl-3.0) 
 >This branch corresponds to the results of the crawler for the study linked with [BioFlow-Insight](https://gitlab.liris.cnrs.fr/sharefair/bioflow-insight)
-[![MIT licensed](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE) 
+## Results of the crawler
+The crawler gathered 677 open license Nextflow workflows. A static version of this corpus can be found here: [https://zenodo.org/records/10817606](https://zenodo.org/records/10817606).
 ## Description
@@ -22,6 +29,7 @@ While the crawler is running, the data is saved in a JSON file.
 ## Table of Contents
 - [Github-Crawler](#github-crawler)
+  - [Results of the crawler](#results-of-the-crawler)
  - [Description](#description)
  - [Table of Contents](#table-of-contents)
  - [Installation](#installation)
@@ -33,7 +41,9 @@ The python function dependancies are described in the `requirements.txt` file.
 ## License
-This project is licensed under the [MIT License](https://opensource.org/licenses/MIT)
+This project is licensed under the [GNU Affero General Public License](https://www.gnu.org/licenses/agpl-3.0.en.html).
 ___

--- a/analysis_results.ipynb
+++ b/analysis_results.ipynb
@@ -76,13 +76,13 @@
     "name": "stdout",
     "output_type": "stream",
     "text": [
-      "Hence, at least 93.1% of Nextflow workflows found on Github are not integrated into WorkflowHub\n"
+      "Hence, at most 6.9% of Nextflow workflows found on Github are integrated into WorkflowHub\n"
     ]
    }
   ],
   "source": [
    "nb_wfhub = 52\n",
-    "print(f\"Hence, at least {(len(dict)-nb_wfhub)/len(dict)*100:.1f}% of Nextflow workflows found on Github are not integrated into WorkflowHub\")"
+    "print(f\"Hence, at most {(nb_wfhub)/len(dict)*100:.1f}% of Nextflow workflows found on Github are integrated into WorkflowHub\")"
   ]
  },
  {

 %% Cell type:markdown id: tags:
 # Analysis of results of crawler
 %% Cell type:code id: tags:
 ``` python
 import seaborn as sns
 import matplotlib.pyplot as plt
 import numpy as np
 sns.set(style = 'darkgrid', palette = "Accent")
 taille = (9, 5)
 ```
 %% Output
    /usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.1
      warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
 %% Cell type:code id: tags:
 ``` python
 import json
 import pandas as pd
 with open('wf_crawl_nextflow.json') as json_file:
    dict = json.load(json_file)
 _ = dict.pop("last_date")
 ```
 %% Cell type:code id: tags:
 ``` python
 print(f"The crawler found {len(dict)} Nextflow workflows with at least Nextflow file at the root.")
 ```
 %% Output
    The crawler found 752 Nextflow workflows with at least Nextflow file at the root.
 %% Cell type:markdown id: tags:
 At the time of writing there are 52 Nextflow workflows integrated on WorkflowHub.
 %% Cell type:code id: tags:
 ``` python
 nb_wfhub = 52
-print(f"Hence, at least {(len(dict)-nb_wfhub)/len(dict)*100:.1f}% of Nextflow workflows found on Github are not integrated into WorkflowHub")
+print(f"Hence, at most {(nb_wfhub)/len(dict)*100:.1f}% of Nextflow workflows found on Github are integrated into WorkflowHub")
 ```
 %% Output
-    Hence, at least 93.1% of Nextflow workflows found on Github are not integrated into WorkflowHub
+    Hence, at most 6.9% of Nextflow workflows found on Github are integrated into WorkflowHub
 %% Cell type:code id: tags:
 ``` python
 tab = []
 for wf in dict:
    tab.append(dict[wf])
 df = pd.DataFrame(tab)
 ```
 %% Cell type:code id: tags:
 ``` python
 name = "Nextflow workflows"
 df[name] = 1
 df["First commit"] = pd.to_datetime(df["first_commit_date"], format=f'%Y-%m-%d')
 fig = plt.figure()
 ax = fig.add_subplot(111)
 ax2 = ax.twinx()
 group_by = 'First commit'
 df[group_by].hist(ax=ax, bins=84)
 df[[name, group_by]].groupby(group_by).count().cumsum().plot(linewidth=2, color='orangered', ax=ax2, grid=False)
 ax.set_ylabel('Counts')
 ax2.set_ylabel('Cumulative sum')
 ax.set_xlabel('Date')
 plt.title("Evolution of the yearly and cumulative number of Nextflow workflows available on GitHub");
 plt.show()
 ```
 %% Output
 %% Cell type:markdown id: tags:
 We only want to use open workflows, so we are only gonna use the workflows which have an open license. We are gonna keep the ones which have :
 * Apache License 2.0
 * GNU General Public License v3.0
 * MIT License
 %% Cell type:code id: tags:
 ``` python
 nb_open = len(df[(df["license"] =="Apache License 2.0") | (df["license"] == "GNU General Public License v3.0") | (df["license"] == "MIT License")])
 print(f"Only keeping open workflows that leaves us with {nb_open} workflows.")
 ```
 %% Output
    Only keeping open workflows that leaves us with 677 workflows.
 %% Cell type:code id: tags:
 ``` python
 ```