This commit updates the shadow free invariant image to 2 colors from
just grayscale.
If we need to save ROM room in the future we'll just disable the LUT and
have the algorithm run with the regular C code. Right now this is not an
issue.
Someone asked me about doing a field of receptors before. These scripts
show how to do that. Also, added example scripts for calling the linear
polar and log polar methods added previously which power
find_rotscale().
Just doing one big commit/PR here since I noticed that breaking it up
causes issues.
Anyway, these fixes give us GOOD/WORKING/FAST optical flow now on the
OpenMV Cam M7. A number of changes were made to the optical flow
scripts. You have have absolute and differential estimation example
scripts. Additionally, you also have the ability to measure rotation and
scale changes too. Linear/Log Polar conversion was added for this. Users
may use the new code for generic image manipulation too. Finally, I
updated the power of 2 resolutions since you actually HAVE to use them
with optical flow for the phasecorrelation code to work correctly.
I have some more advanced scripts coming after this. But, This commit is
already getting kinda large so I'm stopping it here.
* Added hmirror and vflip support to the MT9V034 and example scripts.
* Moved sensor example scripts to one place.
* Add delay to these script for register settling time.
* Textual register cleanup.
No functional changes.
* Add exposure control support.
You can now set the exposure for the camera in microseconds (versus an
opaque unknown value previously). First, we have a new method called
get_exposure_us() which will get the expsoure time in microseconds. This
let's you determine what the auto exposure algorithm set the exposure
time to. Second, the previously implmented set_auto_exposure() method
which allows you to turn aec off and on accepts a exposure_us keyword
argument when you turn aec off to manually control the exposure.
The next commit will add support for other sensor types.
* Cleanup register formatting.
No functional changes.
* Add exposure control support for the OV2640.
Register access for this chip is a PITA.
* Formatting Cleanup.
No functional changes.
* Add exposure control for ov9650.
Just doing it for all sensors.
* Add missing 2 factor.
* Added exposure control for the MT9V034.
* Add exposure control example.
Works well on the OV7725.
Just updating the code with the same style as other methods. I have
another new sister method for histeq() comming up next which I'll push
as soon as this PR is done. Didn't want to merge the two into one PR.
This fix allows "copy_to_fb" with a different resolution than the
current frame buffer to work. It also allows the frame buffer to be
resized, etc. In particular, the pooling methods I added for optical
flow work again... you'll also be able to scale the frame buffer too.
You can now allocate an extra frame buffer for storing images. However,
this takes memory from the main frame buffer. In particular this reduces
the RAM for many methods that do image processing making memory errors
more likely to happen. Note that you may allocate as many extra fb's as
you like. Dealloc happens in reverse order.
Anyway, you can use this method to now storage things like difference
images in RAM allowing for MUCH faster frame difference image
processing.
Moving on, to keep memory management sane... the second fb looks just
like an image and you can use all the image methods to load and update
it, etc. That said, if users deallocate the second FB they need to *NOT*
use the image pointer anymore. There's no way for me to delete the image
pointer in python right now so this is just something that has to be
manually managed (even if I did setup a deconstructor the second FB is
on a stack... so, things wouldn't work so easily with that).
It's now faster to be more useful.
Need to work on HDR for the sensor and making the sensor output better.
I fixed some issues with the illuminvar() method going crazy when it
gets colors with values near 0... but, the shot noise from the sensor
adds a lot of noise to everything. Fixing this will likely solve a lot
of algorithm problems.
Image comparison using SSIM. It can be used to detect image
differences... but, the algorithm was designed to compare image quality
and look at compression artifacts. Anyway, it works kinda okay for
detecting frame differences.
Both algorithms were tested on the OpenMV Cam using images loaded from a
file and work correctly. However, shot noise from the sensor.snapshot()
makes the output value somewhat worthless except in a situation unless
you've controlled for it. Anyway, the illuminvar work best when the
image is constrained to a very particular view point looking at a flat
scene without shadow and then a shadow enters.
(Not adding demo's for these methods since the output looks like crap
unless you've put some work into constraining the scene... need to add
HDR code and other stuff to the sensor module to get better images).
regression code for racing.
No more memcpys all over the place. Not sure why I was doing that.
... code must have been written by an idiot before :) (me).
* The following issues still need fixing:
* Al fb_alloc nlr hooks are DISABLED.
* modnetwork causes cam to hardfault.
* Had to reduce heap by 1K (vfs buffer had to be moved to bss/data).
* self-tests are disabled (cam gets stuck after executing).
Now you can find circles with your OpenMV Cam! The alrogithm can eek out
about 7 FPS on a 160x120 image which is quite impressive given how
computationally expensive circle finding is...
For easy line following mainly. In non-robust mode the line is computed
using least squares. In robust mode the line is computed using the
Theil-Sen median of slopes method. We do not use the Siegel Median of
Medians operation because it costs more CPU time... but, more
importantly there's no way to improve the centroid estimate so even if
the slope is more robust the line will be drawn in the wrong place.
These two new classes allow you to record image data for later viewing
at the same speed the image data was recorded. Unlike GIF/MJPEG the
image data is stored on the file system completely uncompressed in
native frame buffer format making super fast reading and writing
possible. Recording VGA Grayscale at ~13 FPS is possible along with
playing it back. (That's about 30 Mb/s folks).
...
The motivation for writing these scripts is so that you can record video
of something like a line following track, take that video home, and work
on computer vision algorithms for that data.
These classes should make it a lot easier to use the camera at home now.
Moved structs along with image copying code from sensor into
framebuffer.c so that we can use the new copy_fb_to_jpeg_fb() function
in the image library for methods with "copy_to_fb" so that they update
the IDE preview when called.
Also, I noticed that the MAIN_FB_SIZE() value is not calculated
correctly in all cases. Will fix later. Trying to keep this commit clean
for just the refactoring.
All changes have been tested. Too.
With the new frame rate speed increase folks will be asking for smaller
resolutions to get 85 FPS or so when running an algorithm. This commit
adds all scaled modes of frame sizes we already support. We should be
good now on frame sizes for the present and future now.
Todo - skip frames does not run long enough anymore for auto white
balance and gain to stablize before they are turned off in some scripts.
This needs to be adjusted.
Frame rate now can hit 30 FPS when JPEG compression is off. Merging of
lines is perfected too which greatly reduces the noise output. Also,
lines are now objects so you can get their values in an easy way.
We now have a nice and fast malloc system that easily offers 300KB+
dynamic memory... No need to use xalloc anymore except when we're
transfering objects to MP memory space.
The user can now call compressed_for_ide() and compress_for_ide() on an
image to make a jpeg compressed image formatted for transmission over a
data link other than USB. Note that OpenMV IDE will automatically handle
one of these compressed images ending up in the frame buffer and display
it like normal.
To send the image data the user can do:
print(img.compress_for_ide(), end='')
print(img.compressed_for_ide(), end='')
uart.write(img.compress_for_ide())
uart.write(img.compressed_for_ide())
and etc. As mentioned above, compress() compresses the image in place.
And that in place compressed image will then end up in the jpeg buffer.
OpenMV IDE will automatically handling decoding these special compressed
images when this happens.
All variations of the above code have been tested and are working.
Main ZBar code, breaking the commit up because the main file is big.
I will refeactor UMM alloc out of apriltag.c and zbar.c once I'm
finished with this commit stream.
ZBar integration gives us support for basically all 1D linear barcodes.
* Delay the FB size check and corrections to snapshot(). If the frame doesn't
fit FB it gets cropped for GS, or the sensor is switched to bayer for RGB.
Everything works. Running out of memory is fixed and the rotation value
is valid now. For 320x240 operation on the STM32H7 we're going to need
on the order of 1 MB in the entire frame buffer. The code is designed to
handle us getting this amount of memory without any new changes for
320x240 support.
This file includes all of the relevant header/source files from the
april tag library merged into one big file. Additionally, it also
includes heap/quicksort code. I've done the work of going through
the april tag library line by line and fixing it to use fb_alloc,
floats, and our fast math functions.
Anyway, I'm sending this massive file by itself first since it's so
big. Note that we migh in the future want to pull things out of this
file for our own use later if we need linear algebra support.
I also tested the firmware for about an hour to make sure there was no
stack leak.
Note that I prefer for fb_free() to still be called versus
fb_free_till_mark() doing that for you in the code.
For functions without this fix they will just free the entire fb_alloc
stack when an exception happens. For functions with this fix they will
only free up to and including the mark. Since there are no places in the
firmware where you could start building a second fb_alloc stack when one
is already in place this point is moot currently. But, if we do
something like that in the future the problem will have already been
solved.
Any new code or re-worked code should use the mark function.
Speed up the algorithm by fixing the abs() issue. Do not use that
function in any of your code. It by itself cut the speed of the code
in half. I don't know what's in that function but I'm guessing it does
ABS of a float using ints or something.
I made the zoom parameter functional now too so you can use lens_corr to
zoom in on the image. Argument parsing is handled too. Finally, I
updated the only script where this is used.
Note that I'm able to get more than 10 FPS at 160x120 on the M4 and 15
FPS at 160x120 on the M7. Previous this was at about 5 FPS and 7.5 FPS
respectively.