OpenCV supports reading of images in
most file formats such as JPEGs, PNGs, and TIFF.
Most image and video analysis requires
converting images into gray scale first.
This simplifies the image and reduces
noise while allowing for improved analysis.
Let's write some code that reads in an image
of a person Floyd Mayweather,
and converts it into gray scale.
So first we're going to import the openCV package cv2.
So import cv2 as cv and then
we'll load the Floyd dot JPEG image.
So image equals cv dot read from
read-only Floyd dot JPEG and we'll
convert it to gray scale using this cv color image.
So gray equals cv dot cvtColor.
This is function in openCV passing
the image and then cv dot color BGR2GRAY.
Now, before we get to
the result let's talk about the docs.
Just like tesseract,
OpenCV is an external package written
in C plus plus and the docs for Python are really poor.
This is unfortunately quite common
when Python is being used as a wrapper.
Thankfully, the web docs for
OpenCV are actually pretty good.
So hit the website at docs dot opencv
dot org when you want to learn
more about a particular function.
In this case, cvtColor
converts from one color space to another,
and we're converting our image to gray scale.
Of course, we already know
at least two different ways of doing this
using binarization and PIL color space conversions.
So let's inspect this object that has been returned.
So import inspect, this is in the standard library
inspect dot get mro and then type of gray.
So we see that this is a type of ndarray which is
a fundamental list type coming from
the numerical Python project.
That's a bit surprising.
Up until this point we've been used to working
with these PIL dot Image objects.
OpenCV however, wants to represent
an image as a two dimensional sequence of bytes,
and the ndarray which stands for an n dimensional array,
is the ideal way to do this.
Let's look at the contents of the array, so just gray.
So the array shown here is a list of lists
where the inner lists are filled with integers.
The d type equals u int eight
definition indicates that each of the items
in the array is an eight bit unsigned integer
which is very common for black and white images.
So this is a pixel by pixel definition of the image.
The display package however,
doesn't know what to do with this image.
So let's convert it into
a PIL object to render it into the browser.
So from PIL import image,
PIL can take an array of data with
a given color format and convert this into a PIL object,
and this is perfect for our situation
as the PIL color mode,
L, is just a ray of luminate values in unsigned integers.
So the new image equals image dot from array,
gray and then we tell it
luminance values and then display image.
Let's talk a bit more about images for a minute.
Numpy arrays are multi-dimensional.
For instance, we can define
an array in a single dimension.
So import numpy as
np and then single dim equals np dot array
and we just pass into
this a list of all of the integer values.
In an image, this is analogous to
a single row of five pixels each in gray scale.
But actually all imaging libraries tend
to expect at least two dimensions,
a width and a height,
and to show a matrix.
So if we put the single dim inside of
another array this would be a two-dimensional array
with the equivalent of
one element in the height direction
and five in the width.
So we can go double dim equals np dot array,
and here we just taking a single dimension array and
put it inside another bracket,
and so double dim.
So this should look pretty familiar.
It's a lot like the list of lists we saw above.
Let's see what this new two-dimensional array looks
like if we actually display it to the screen.
So display and then we have to convert to
PIL so image dot from array we pass
in the double dim and say that
it's a luminance and this
should display it to the screen.
So that's pretty unexciting hits just a little line.
It's actually five pixels in a row to be
exact of different levels of black.
The numpy library has a nice attribute called shape that
allows us to see how many dimensions big an array is.
The shape attribute returns a tuple that
shows the height of the image by the width of the image.
So double dim but dot shape.
Let's take a look at the shape of
our initial image that we loaded into the image variable.
So img dot shape.
This image has three dimensions,
that's because it's got a width a height
and what's called a color depth.
In this case the color is
represented as an array of three values.
Let's take a look at the color of the first pixel.
So the first pixel is equal to image
sub-zero and the width sub-zero on the height,
and so the first pixel.
Here we see that the color value is provided in
full RGB using an unsigned integer.
This means that each color can have one of
256 different values and
that the total number of unique colors
that can be represented by this data is 256 by
256 by 256 which is roughly 16 million colors.
We call this 24-bit color
which is eight plus eight plus eight, 24.
If you find yourself shopping for
a television you might notice that
some expensive models are advertised as having
10 bit or even 12-bit panels.
These are televisions where each of
the red green and blue color channels are
represented by 10 or 12 bits instead of eight.
For 10 bit panels,
this means that there's over one billion colors
capable and 12-bit panels are
capable of over 68 billion colors.
We're not going to talk much more about color in
this course but it's a fun subject.
Instead, let's go back to this array representation of
images because we can do
some interesting things with this.
One of the most common things we can do with
an ndarray is to reshape it,
to change the number of rows and columns that are
represented here so that we
could do different kinds of operations.
Here's our original two-dimensional image.
So let's print original image and then print out gray.
If we wanted to represent that as
a one-dimensional image we just call reshape,
so print new image and reshape takes
the image as the first parameter
and a new shape as the second.
So let's say image one d equals np dot reshape we
send in our original array gray and then the new shape
will have as one and then
gray dot shape times gray dot shapes up
zero and one multiply them together you get
the total number of pixels and then
print image one d. So why are we
talking about these nested arrays of bytes when we're
supposed to be talking about OpenCV as a library?
Well, I wanted to show you that often
libraries work on the same kind of principals.
In this case that images are stored as arrays of
bytes and they're not
representing the data in the same way in their APIs,
but by exploring a bit you can learn
how the internal representation of
data is stored and build
routines to convert between formats.
For instance, remember the last lecture
when we wanted to look for
gaps in an image so we could
draw lines to feed into cracking.
Well, we use PIL to do this using
the get pixels to look at the individual pixels and see
what luminosity values were
then image dot draw dot rectangle to
actually fill in a black bar separator
and this was a nice high-level API.
Let us write some routines to
do the work we wanted without having to
understand too much about
how the images were actually being
stored but computationally it was very slow.
Instead, we could write the code to
use this using matrix features within numpy.
Lets take a look, so import cv2 as
cv will load the two column image as well so image
equals cv dot read and read
only two col and we'll convert it to gray-scale
using this cvtColor image so gray equals cv dot
cvt color image and color two gray.
Now, remember how slicing on a list works.
If you have a list of a number such as
a equals zero through five and then you
go a sub two colon four that's going to
return a list of numbers at
positions two through four inclusive,
and don't forget that lists start indexing at zero.
If we have a two-dimensional array we can slice out
a smaller piece of that using the format a sub
two comma four for the first dimension
and then one colon three for
the second dimension and you could think of this as first
slicing along the row dimension
then in the column's dimension.
So in this example that would be a matrix of rows
two and three and columns one and two.
Here's a look at our image
so gray two colon four and one colon three.
So we see that it's all white.
We can use this technique as
a window and move it around our big image.
Finally, the ndarray library has lots of
matrix functions which are generally very fast to run,
one that we might want to consider in this case is
count non-zero which just returns
the number of entries in the matrix which are not zero
so np dot count non-zero we can
say gray and give it two colon
four and one colon three and so this is going
to crop out essentially a piece of the image
and as a matrix and send it to np dot count non-zero.
Okay, the last benefit of
going to this low-level approach
to images and so we can change pixels very fast as well.
Previously we were drawing rectangles and
setting a fill and a line width,
this is nice if you want to do
something like change the color of the fill
from the line or draw complex shapes like polygons,
but we really just wanted a line
here and that's really easy to do,
all we have to do is change the number of
luminosity values from 255 to zero.
Here's an example, let's create a big white matrix.
So white matrix equals and we'll use
np dot four we wanted a 12 by
12 matrix as we'll pass that in as the tuple
for the shape of it 255 because
we want everything to be white and we'll
set the d type to be np dot
u int eight so that means
a one byte per pixel to describe it.
Now let's display that.
So remember we have to do image dot
from array to convert from PIL
passing the white matrix tuple
that it's luminosity values,
and let's print out the white matrix as well.
So this looks pretty boring,
it's just a giant white square
which we actually can't see
but if we want we can easily
color our column to be black.
So if we go white matrix sub and here I just
want all row values so I just use
a colon comma six so I want the sixth,
well in this case the seventh column,
and I want to set this to be np dot
four and I just want to add in
a matrix of one colon 12 and
shape with zero that's also d type int,
and I want to display this
to the screen so image dot from
array white matrix which now won't just be
white and then print out the white matrix.
That's essentially what we wanted to do so,
why should we do it this way when it seems
so much more low level and
really the answer speed this paradigm of using
matrices to store and manipulate
bytes of data for images is much closer
to how low-level API and
hardware developers think about
storing files and bytes and memory.
How much faster is this?
Well, that's up to you to discover.
There is an optional assignment for this week
to convert our old code over into
this new format to compare
both the readability and
the speed of the two different approaches.