diff --git a/README.md b/README.md index c5bd26ded59e93c3799bdbc82c8c0846a32dd926..d0b39e6503414f751fa837719d87b864558e81c4 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,98 @@ +<h1 style="text-align:center"> +Synthetic Driver Image Generation for Human Pose-Related Tasks +</h1> +<div style="text-align:center"> +<h3> +<a href="https://liris.cnrs.fr/page-membre/romain-guesdon">Romain Guesdon</a>, +<a href="https://liris.cnrs.fr/page-membre/carlos-crispim-junior">Carlos Crispim-Junior</a>, +<a href="https://liris.cnrs.fr/page-membre/laure-tougne">Laure Tougne Rodet</a> +<br> +<br> +International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP) +</h3> +</div> + +# Table of content +- [Overview](#overview) +- [Installation](#installation) +- [Citation](#citation) +- [Acknowledgements](#acknowledgements) + +# Overview +This repository contains the materials presented in the paper +[Synthetic Driver Image Generation for Human Pose-Related Tasks](). +We provide scripts and guidance to generate your own images along with a link to download our 200k driver images dataset. + + +# Installation +## Requirements +#### Blender +This script has been developed and ran using Blender 3.2.0. Blender can be donwloaded [here](https://www.blender.org/download/). +#### MakeHuman +Human models were generated using [MakeHuman Community](http://www.makehumancommunity.org/content/downloads.html). We used a modified skeleton rig that you can find in the `mh_models/` directory of this repo. Once MakeHuman is installed, copy the two "blender.*" files in `MAKEHUMAN-COMMUNITY-ROOT\makehuman\data\rigs`. +We also highly recommand to install [MassProduce](http://www.makehumancommunity.org/content/plugins.html) plugin to generate rapidely several models. + +## Models +#### Human models +To generate human models using MakeHuman, first set: +- in Files > Export : Mesh format = Makehuman Exchange (.mhx2), Scale Units = meter +- in Pose / Animate > Skeleton > Rig presets = Blender + +Then, you can either mannualy modified your model or use MassProduce in the Community tab after setting all the random parameters, and export it as MHX2. +By default, you should place the mhx2 files in `mh_models/exports` and the associated texture files in `mh_models/exports/textures`. + +#### Cars models +You can use 3D car models you got. Car models should be imported in an empty Blender file, with its location and rotation set to 0 and a real-size scale. +The origin of the model should be placed on the driver since it will be used to place the driver. You can set an Empty Arrows object as the parent in Blender to achieve this goal, like below : +<img src="./car_origin_blender.png" alt= "car_origin_blender" width="200"/> + +Furthermore, you can add empty meshes such as cylinders or cubes to force the hand to be placed inside or outside. To do this, rename them "IN_..." or "OUT_...". We recommand placing a "OUT_" cylinder around the wheel, and a "IN_" box to delimit the inside of the car cockpit. + +Finally, you can put target for the wrists that will be used for the manual poses. To do this, place Empty Arrows in the desired location and rotatio, with the name "Target_R_..." or "Target_L_...", such as: +<img src="./target_examples.png" alt= "target_examples" height="200"/> + +# Valid visibility module +You can download pretrained weights [here](http://dionysos.univ-lyon2.fr/~ccrispim/VisPred/models). +For example, run: +``` +python pose_estimation/valid.py \ + --cfg experiments/coco/resnet50/256x192_vis_freeze.yaml \ + --flip-test \ + --model-file models/pytorch/pose_coco/coco_vis2_raise_soft.pth.tar +``` + + + + +# Citation +If you use our network or our code, please cite: +``` +@InProceedings{Guesdon_2023_Visapp, + author = {Guesdon, Romain and Crispim-Junior, Carlos and Tougne Rodet, Laure}, + title = {Synthetic Driver Image Generation for Human Pose-Related Tasks}, + booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - To be published,}, + year={2023}, + pages={}, + publisher={SciTePress}, + organization={INSTICC}, +} +``` + +# Acknowledgments +This work was supported by the Pack Ambition Recherche 2019 funding of the French AURA Region in +the context of the AutoBehave project. +<div style="text-align:center"> +<img style="margin-right: 20px" src="assets/logo_liris.png" alt="LIRIS logo" height="75" width="160"/> +<img style="margin-left: 20px" src="assets/logo_ra.png" alt="RA logo" height="60" width="262"/> +</div> + + + + # Synthe_DriPE ### Addons Blender: MHX2 : http://download.tuxfamily.org/makehuman/plugins/mhx-blender-latest.zip Import images as planes -Lighting : Sun position \ No newline at end of file +Lighting : Sun position diff --git a/car_origin_blender.png b/car_origin_blender.png new file mode 100644 index 0000000000000000000000000000000000000000..2be5d873b1ac311282904ea231ba56e4821f09a2 Binary files /dev/null and b/car_origin_blender.png differ diff --git a/target_examples.png b/target_examples.png new file mode 100644 index 0000000000000000000000000000000000000000..2b1bbd0dd81b1cd62a063dcecad761cc456a6e20 Binary files /dev/null and b/target_examples.png differ