1
00:00:07,880 --> 00:00:10,050
One of the nice things about

2
00:00:10,050 --> 00:00:11,790
using the Jupyter
Notebooks system,

3
00:00:11,790 --> 00:00:13,200
is that there's a rich set of

4
00:00:13,200 --> 00:00:16,425
contributed plugins that
seek to extend this system.

5
00:00:16,425 --> 00:00:18,450
In this lecture, I want
to introduce you to

6
00:00:18,450 --> 00:00:21,965
one such plugin
called ipy web rtc.

7
00:00:21,965 --> 00:00:24,870
Web rtc is a fairly new protocol

8
00:00:24,870 --> 00:00:27,420
for real-time
communication on the web.

9
00:00:27,420 --> 00:00:29,400
I'm talking about chatting.

10
00:00:29,400 --> 00:00:33,020
The widget brings this to
Jupyter notebook systems.

11
00:00:33,020 --> 00:00:35,360
Let's take a look. First, let's

12
00:00:35,360 --> 00:00:36,650
import this from the library,

13
00:00:36,650 --> 00:00:39,250
two different classes which
we're going to use in a demo.

14
00:00:39,250 --> 00:00:41,255
One for the camera and
one for the images.

15
00:00:41,255 --> 00:00:43,555
So from ipy web rtc,

16
00:00:43,555 --> 00:00:46,705
import camera stream
and image recorder.

17
00:00:46,705 --> 00:00:49,910
Then let's take a look at
the camera stream objects,

18
00:00:49,910 --> 00:00:51,895
or help camera stream.

19
00:00:51,895 --> 00:00:54,410
We see from the docs
that it's easy to

20
00:00:54,410 --> 00:00:56,585
get a camera facing the user.

21
00:00:56,585 --> 00:00:58,895
Then we have the audio on or off.

22
00:00:58,895 --> 00:01:00,590
We don't need audio
for this demo so

23
00:01:00,590 --> 00:01:02,480
let's create a new
camera instance.

24
00:01:02,480 --> 00:01:05,150
So camera equals
cameras stream.facing

25
00:01:05,150 --> 00:01:08,240
user and audio equals false.

26
00:01:08,240 --> 00:01:11,914
The next object we want to
look at is the image recorder,

27
00:01:11,914 --> 00:01:14,640
so help image recorder.

28
00:01:14,800 --> 00:01:17,600
The image recorder
let us actually

29
00:01:17,600 --> 00:01:19,940
grab images from
the camera stream.

30
00:01:19,940 --> 00:01:21,590
There are features
for downloading

31
00:01:21,590 --> 00:01:23,090
and using the image as well.

32
00:01:23,090 --> 00:01:26,180
We see that the default format
is a ping file.

33
00:01:26,180 --> 00:01:28,655
Let's look up the image
recorder to our stream.

34
00:01:28,655 --> 00:01:30,560
So image recorder equals

35
00:01:30,560 --> 00:01:33,550
image recorder and stream
is equal to the camera.

36
00:01:33,550 --> 00:01:35,360
Now, the dogs are a little

37
00:01:35,360 --> 00:01:37,940
unclear about how you
use this within Jupyter.

38
00:01:37,940 --> 00:01:40,145
But if we call
the download function,

39
00:01:40,145 --> 00:01:42,620
it will actually store
the results of the camera,

40
00:01:42,620 --> 00:01:44,240
which is hooked up in image

41
00:01:44,240 --> 00:01:47,095
recorder.image. Let's try it out.

42
00:01:47,095 --> 00:01:49,310
First, let's tell
the image recorder

43
00:01:49,310 --> 00:01:50,690
to start capturing data.

44
00:01:50,690 --> 00:01:53,915
So image reporter.recording
equals true.

45
00:01:53,915 --> 00:01:56,030
Now, let's download the image,

46
00:01:56,030 --> 00:01:57,800
so image recorder.download.

47
00:01:57,800 --> 00:02:00,860
Then, let's inspect
the type of the image,

48
00:02:00,860 --> 00:02:04,700
so type, image recorder.image.

49
00:02:04,700 --> 00:02:07,790
Okay. So the object
that it's stored

50
00:02:07,790 --> 00:02:11,900
as an ipywidgets.widget.widget
media.image.

51
00:02:11,900 --> 00:02:14,495
How did we do something
useful with this?

52
00:02:14,495 --> 00:02:17,090
Well, an inspection of
the object shows that there's

53
00:02:17,090 --> 00:02:18,755
a handy value field which

54
00:02:18,755 --> 00:02:21,395
actually holds the bytes
behind the image,

55
00:02:21,395 --> 00:02:23,405
and we know how to display those.

56
00:02:23,405 --> 00:02:27,380
So let's import PIL
image import PIL.Image.

57
00:02:27,380 --> 00:02:30,440
Now, let's import IO import IO.

58
00:02:30,440 --> 00:02:33,140
Now, let's create
a PIL image from the bytes.

59
00:02:33,140 --> 00:02:36,785
So images equal to
pil.Image.open, and then,

60
00:02:36,785 --> 00:02:39,380
io.BytesIO to create a stream,

61
00:02:39,380 --> 00:02:42,380
and then image
recorder.image.value.

62
00:02:42,380 --> 00:02:44,410
Wow. That's a lot of dots.

63
00:02:44,410 --> 00:02:46,730
Let's render it to the screen.

64
00:02:46,730 --> 00:02:48,530
So that's just display image,

65
00:02:48,530 --> 00:02:51,005
because it's just
a PIL image down.

66
00:02:51,005 --> 00:02:53,390
Great. You see a picture.

67
00:02:53,390 --> 00:02:55,250
Hopefully, you're
following along and one of

68
00:02:55,250 --> 00:02:56,630
the note bugs and
you've been able to

69
00:02:56,630 --> 00:02:58,415
try this out for yourself.

70
00:02:58,415 --> 00:03:00,650
So what can you do with this?

71
00:03:00,650 --> 00:03:02,210
Well, this is a great way to get

72
00:03:02,210 --> 00:03:04,055
started with a better
computer vision.

73
00:03:04,055 --> 00:03:05,600
You already know how to identify

74
00:03:05,600 --> 00:03:07,880
a face and a webcam
in the picture,

75
00:03:07,880 --> 00:03:10,475
or try and capture texts
from within the picture.

76
00:03:10,475 --> 00:03:11,870
With Open CV, there's

77
00:03:11,870 --> 00:03:13,985
any number of things
that you can do.

78
00:03:13,985 --> 00:03:15,395
Simply with a webcam,

79
00:03:15,395 --> 00:03:18,390
the Jupyter notebooks and Python.