EyeTrackVR项目的固件
Go to file
2022-03-28 01:13:06 +02:00
.idea Add basic esp32cam webserver from examples 2022-02-24 01:04:56 +01:00
docs Refactor PoC code into namespaces, add led management, improve directory structure (#2) 2022-02-27 19:20:10 +01:00
ESP Add MDNS support (#9) 2022-03-28 00:03:29 +02:00
README.md Fixup readme formatting vol2 2022-03-28 01:13:06 +02:00

#OpenIris OpenIris is the firmware part of the EyeTrackVR Project.

The aim of this project is to provide a fast and performant firmware for streaming the eye data back to the PC for further processing and actual tracking.

NOTE

This is a work in progress project, things in the main branch are tested and should work just fine, but as it is in its very early stages it might not "just work".

Nonetheless, pull requests and issues are welcome, let's get this cracking!

Features

Working right now

  • Basic stream in 60FPS at 248x248px in MJPEG in greyscale
  • A basic HTTP server with API
  • Basic control of the camera though API
  • Health checks
  • OTA updates
  • ROI selection for eye area

TODO

  • Implementation of littleFS for saving camera and ROI settings
  • MDNS - so that the server itself will detect and communicate with the tracker without you doing anything. No need to configure IPs and stuff, it's automagic
  • LED status patterns - so that you know what's going on without plugging the tracker in to the PC
  • FEC encoding with packet injection for even faster streams!
  • Better OTA so that updates can be downloaded from github and pushed by the server to the tracker
  • CI/CD with github actions - so we can more seamlessly update the trackers

Accompanying projects

EyeTrackVR - The core project, made by Prohurtz

IndexEyeTrackVR - An OpenCV version of the EyeTrackVR with a mount for Valve Index made by one of the community members - Razgriz!

VrEyeToolbox - A compilation of different AI/ML methods and models for eye tracking in VR, made by Summer from CFVR!

Additional equipment and software:

Below is a list of parts, mounts, etc

Electronics

Currently, the following hardware is being used for testing. There is no guarantee these will be the final parts, and everything is heavily in flux. Links to AliExpress and Digikey listings for some items are included - these are for reference, and do not represent endorsements.

Mounts

IndexEyeTrackVR - A PoC version of an index mount made by Razgriz

Mounts for the Oculus Quest 2.

Now, how do I get it running?

In the future, all you would have to do would be to push a flash/update button in the server and everything would be set up.

But we're not there yet so here's how to get it running manually:

  1. Clone this repo
  2. Download VSCode
  3. Install platformio IDE plugin for VSC
  4. Install the drivers for CH341SER from here
  5. Open the cloned project in VSC
  6. Set your own credentials in ESP/include/credentials.h NOTE: special characters are not supported by the ESP
  7. Connect the esp to the pc via a micro usb cable. Make sure that it can transfer data, otherwise this will not work
  8. Click on the Ant's head icon on the left and from the menu select build, wait till it finishes and then click on upload and monitor

Aight, I want to help, what should I do?

That's great to hear, I'm not a cpp dev, nor am I good with embedded so help is more than welcome. That being said:

  1. Fork and clone the project
  2. Set up Visual Studio Code or Clion with the platformio plugin
  3. Take a look at the project board - that's where I note everything I'm working on, I've finished and what I'm planning to do
  4. Make a PR and / or ping me in the EyeTrackVR discord

Questions, help, bugs, suggestions, blames?

Ping me in the EyeTrackVR discord, I'm - lorow#6600