Let's start talking about logistic regression.
In this video, I'd like to
show you the hypothesis representation, that
is, what is the function
we're going to use to represent
our hypothesis where we have a classification problem.
Earlier, we said that we
would like our classifier to
output values that are between
zero and one. So, we
like to come up with a
hypothesis that satisfies this
property, that these predictions are maybe between zero and one.
When we were using linear regression,
this was the form of a
hypothesis, where H of X
is theta transpose X. For
logistic regression, I'm going
to modify this a little
bit, and make the hypothesis
G of theta transpose X,
where I'm going to define
the function G as follows:
G of Z if Z
is a real number is equal
to one over one plus
E to the negative Z. This
called the sigmoid function
or the logistic function.
And the term logistic function,
that's what give rise to the name logistic progression.
And, by the way, the terms
sigmoid function and logistic
function are basically synonyms
and mean the same thing.
So the two terms are
basically interchangeable and either
term can be used to
refer to this function
G. And if we
take these two equations, and
put them together, then here's
just an alternative way of
writing out the form of my hypothesis.
I'm saying that H of x
is one over one plus
E to the negative theta transpose
X, and all I've done is
I've taken the variable
Z, Z here's a
real number and plugged in
theta transpose X, so
I end up with, you know, theta transpose
X, in place of Z there.
Lastly, let me show you where the sigmoid function looks like.
We're going to plot it on this figure here.
The sigmoid function, G of
Z, also called the logistic function, looks like this.
It starts off near zero and
then rises until it processes
0.5 at the origin and then it flattens out again like so.
So that's what the sigmoid function looks like.
And you notice that the
sigmoid function, well, it
asymptotes at one, and
asymptotes at zero as
Z against the horizontal axis
is Z. As Z goes to
minus infinity, G of
Z approaches zero and as
G of Z approaches infinity, G
of Z approaches 1, and
so because G of
Z offers values that are
between 0 and 1 we
also have that H of
X must be between 0 and 1.
Finally, given this hypothesis
representation, what we
need to do, as before,
is fit the parameters theta to our data.
So given a training set, we
need to pick a value for
the parameters theta and this
hypothesis will then let us make predictions.
We'll talk about a learning algorithm
later for fitting the parameters theta.
But first let's talk a
bit about the interpretation of this model.
Here's how I'm going to
interpret the output of
my hypothesis H of
X. When my hypothesis
outputs some number, I am
going to treat that number as
the estimated probability that Y
is equal to one on a
new input example X. Here is what I mean.
Here is an example.
Let's say we're using the tumor classification example.
So we may have a feature
vector X, which is this x01
as always and then our
one feature is the size of the tumor.
Suppose I have a patient come
in and, you know they have some
tumor size and I
feed their feature vector X
into my hypothesis and suppose
my hypothesis outputs the number 0.7.
I'm going to interpret
my hypothesis as follows.
I'm going to say that this
hypothesis is telling me
that for a patient with
features X, the probability
that Y equals one is 0 .7.
In other words, I'm going
to tell my patient that the
tumor, sadly, has
a 70% chance or a 0.7 chance of being malignant.
To write this out slightly more
formally or to write this
out in math, I'm going to
interpret my hypothesis output
as P of y
equals 1, given X
parametrized by theta.
So, for those of you that are
familiar with probability, this equation
might make sense, if you're a little less familiar
with probability, you know, here's
how I read this expression, this
is the probability that y is
equals to one, given x
instead of given that my patient
has, you know, features X.
Given my patient has a particular
tumor size represented by my
features X, and this
probability is parametrized by theta.
So I'm basically going to count
on my hypothesis to give
me estimates of the probability
that Y is equal to 1.
Now since this is a
classification task, we know
that Y must be either zero or one, right?
Those are the only two values
that Y could possibly take on,
either in the training set or
for new patients that may walk
into my office or into the doctor's office in the future.
So given H of X,
we can therefore compute the probability
that Y is equal to zero as well.
Concretely, because Y must
be either zero or one,
we know that the probability
of Y equals zero, plus the
probability of Y equals
one, must add up to one.
This first equation looks a
little bit more complicated but it's
basically saying that probability of
Y equals zero for a
particular patient with features x, and
you know, given our parameter's theta, plus the
probability of Y equals one for
that same patient which features x and you
parameters theta must add
up to one, if this equation
looks a little bit complicated feel free
to mentally imagine it without that X and theta.
And this is just saying that
the probability of Y equals zero plus
the probability of Y equals one must be equal to one.
And we know this to be
true because Y has to be either zero or one.
And so the chance of Y
being zero plus the chance
that Y is one, you know, those two must add up to one.
And so if you just
take this term and move
it to the right-hand side, then
you end up with this equation
that says probability Y equals zero
is one minus probability y equals
and thus if our
hypothesis if H of X
gives us that term you can
therefore quite simply compute the
probability, or compute the
estimated probability that Y
is equal to zero as well.
So you now know what
the hypothesis representation is for
logistic regression and we're seeing
what the mathematical formula is
defining the hypothesis for logistic regression.
In the next video, I'd like
to try to give you
better intuition about what the
hypothesis function looks like.
And I want to tell
you something called the decision
boundary and we'll look
at some visualizations together to
try to get a better sense
of what this hypothesis function of
logistic regression really looks like.