|
|
||
|---|---|---|
| .idea | ||
| docs | ||
| ESP | ||
| PythonExamples | ||
| README.md | ||
#OpenIris OpenIris is the firmware part of the EyeTrackVR Project.
The aim of this project is to provide a fast and performant firmware for streaming the eye data back to the PC for further processing and actual tracking.
NOTE
This is a work in progress project, things in the main branch are tested and should work just fine, but as it is in its very early stages it might not "just work".
Nonetheless, pull requests and issues are welcome, let's get this cracking!
Features
Working right now
- Basic stream in 60FPS at 248x248px in MJPEG in greyscale
- A basic HTTP server with API
- Basic control of the camera though API
- Health checks
- OTA updates
- ROI selection for eye area
TODO
- Implementation of littleFS for saving camera and ROI settings
- MDNS - so that the server itself will detect and communicate with the tracker without you doing anything. No need to configure IPs and stuff, it's automagic
- LED status patterns - so that you know what's going on without plugging the tracker in to the PC
- FEC encoding with packet injection for even faster streams!
- Better OTA so that updates can be downloaded from github and pushed by the server to the tracker
- CI/CD with github actions - so we can more seamlessly update the trackers
Accompanying projects
EyeTrackVR - The core project, made by Prohurtz
IndexEyeTrackVR - An OpenCV version of the EyeTrackVR with a mount for Valve Index made by one of the community members - Razgriz!
VrEyeToolbox - A compilation of different AI/ML methods and models for eye tracking in VR, made by Summer from CFVR!
Additional equipment and software:
Below is a list of parts, mounts, etc
Electronics
Currently, the following hardware is being used for testing. There is no guarantee these will be the final parts, and everything is heavily in flux. Links to AliExpress and Digikey listings for some items are included - these are for reference, and do not represent endorsements.
- ESP32CAM Development board with ESP32-CAM-MB programmer
- OV2640 Camera Module, 75mm Cable, 160 degree lens, 850nm "Night Vision" - no IR cut filter
- IN-P32ZTIR IR Diode, run at 35 mA - 5V w/ 100ohm resistor
Mounts
IndexEyeTrackVR - A PoC version of an index mount made by Razgriz
Mounts for the Oculus Quest 2.
Now, how do I get it running?
In the future, all you would have to do would be to push a flash/update button in the server and everything would be set up.
But we're not there yet so here's how to get it running manually:
- Clone this repo
- Download VSCode
- Install platformio IDE plugin for VSC
- Install the drivers for CH341SER from here
- Open the cloned project in VSC
- Set your own credentials in ESP/include/credentials.h NOTE: special characters are not supported by the ESP
- Connect the esp to the pc via a micro usb cable. Make sure that it can transfer data, otherwise this will not work
- Click on the Ant's head icon on the left and from the menu select build, wait till it finishes and then click on upload and monitor
Aight, I want to help, what should I do?
That's great to hear, I'm not a cpp dev, nor am I good with embedded so help is more than welcome. That being said:
- Fork and clone the project
- Set up Visual Studio Code or Clion with the platformio plugin
- Take a look at the project board - that's where I note everything I'm working on, I've finished and what I'm planning to do
- Make a PR and / or ping me in the EyeTrackVR discord
Questions, help, bugs, suggestions, blames?
Ping me in the EyeTrackVR discord, I'm - lorow#6600