1
00:00:01,280 --> 00:00:12,559
[Music]

2
00:00:15,120 --> 00:00:18,080
i'm doing this not because i'm

3
00:00:18,080 --> 00:00:21,119
professionally engaged with 360 video

4
00:00:21,119 --> 00:00:24,160
just just an amateur i like taking 360

5
00:00:24,160 --> 00:00:26,640
photos on vacation

6
00:00:26,640 --> 00:00:29,039
and during covet i was thinking about

7
00:00:29,039 --> 00:00:32,320
how to how the experience of

8
00:00:32,320 --> 00:00:33,680
remote video

9
00:00:33,680 --> 00:00:36,480
conferencing could be improved

10
00:00:36,480 --> 00:00:39,920
and that's how i got involved

11
00:00:39,920 --> 00:00:41,440
so um

12
00:00:41,440 --> 00:00:43,840
stream is running right now so if you

13
00:00:43,840 --> 00:00:46,399
want to make sure you're not on camera

14
00:00:46,399 --> 00:00:49,760
consider sitting on that side

15
00:00:49,760 --> 00:00:51,760
yeah so um

16
00:00:51,760 --> 00:00:55,039
live streaming 360 video

17
00:00:55,039 --> 00:00:57,199
is normally a pretty much a no-brainer

18
00:00:57,199 --> 00:00:58,399
you just

19
00:00:58,399 --> 00:00:59,280
you know

20
00:00:59,280 --> 00:01:01,680
if your camera supports it you just

21
00:01:01,680 --> 00:01:04,080
click on it and say i want to stream to

22
00:01:04,080 --> 00:01:06,960
youtube i want to stream to facebook

23
00:01:06,960 --> 00:01:09,119
and

24
00:01:09,119 --> 00:01:11,119
most of the time it works

25
00:01:11,119 --> 00:01:14,000
but i think it's important to do it with

26
00:01:14,000 --> 00:01:16,560
your own infrastructure for the for the

27
00:01:16,560 --> 00:01:18,080
usual reasons

28
00:01:18,080 --> 00:01:21,600
no one is tracking who's watching what

29
00:01:21,600 --> 00:01:23,280
and

30
00:01:23,280 --> 00:01:25,680
also

31
00:01:26,000 --> 00:01:28,080
by using free software you're making

32
00:01:28,080 --> 00:01:30,400
sure that there are no backdoors no

33
00:01:30,400 --> 00:01:33,119
surveillance

34
00:01:33,680 --> 00:01:33,990
and

35
00:01:33,990 --> 00:01:37,110
[Music]

36
00:01:37,280 --> 00:01:38,560
by using

37
00:01:38,560 --> 00:01:41,119
if everyone hosts his own stream it's

38
00:01:41,119 --> 00:01:43,680
decentralized and uh if if one stream

39
00:01:43,680 --> 00:01:45,600
fails then it's

40
00:01:45,600 --> 00:01:47,520
not not everything is down as we've seen

41
00:01:47,520 --> 00:01:51,200
with the large outages of aws and so on

42
00:01:51,200 --> 00:01:52,840
well

43
00:01:52,840 --> 00:01:54,799
um

44
00:01:54,799 --> 00:01:57,439
using free software is unfortunately not

45
00:01:57,439 --> 00:01:59,840
possible end-to-end because the cameras

46
00:01:59,840 --> 00:02:01,680
themselves of course are running some

47
00:02:01,680 --> 00:02:04,399
proprietary firmware at least all those

48
00:02:04,399 --> 00:02:06,399
that i'm aware of

49
00:02:06,399 --> 00:02:09,119
and then some of them can stream

50
00:02:09,119 --> 00:02:12,319
directly and some of them rely on on an

51
00:02:12,319 --> 00:02:14,319
app on your phone or

52
00:02:14,319 --> 00:02:15,280
on

53
00:02:15,280 --> 00:02:18,160
some software running on a pc which is

54
00:02:18,160 --> 00:02:20,640
often also proprietary

55
00:02:20,640 --> 00:02:21,440
but

56
00:02:21,440 --> 00:02:24,800
from that step on in the chain we can

57
00:02:24,800 --> 00:02:27,840
we can put our own stuff in so um i will

58
00:02:27,840 --> 00:02:30,800
cover them during this talk

59
00:02:30,800 --> 00:02:34,000
and the end device is often

60
00:02:34,000 --> 00:02:35,680
also proprietary so if you have like a

61
00:02:35,680 --> 00:02:38,319
vr headset or a phone

62
00:02:38,319 --> 00:02:40,160
not everything running on your phone may

63
00:02:40,160 --> 00:02:41,440
be open source

64
00:02:41,440 --> 00:02:44,480
maybe maybe this maybe it isn't

65
00:02:44,480 --> 00:02:47,760
but um we see how much we can

66
00:02:47,760 --> 00:02:49,360
how much we can do

67
00:02:49,360 --> 00:02:51,440
okay for those of you who are not

68
00:02:51,440 --> 00:02:53,760
familiar with 360 cameras

69
00:02:53,760 --> 00:02:55,440
basically the idea is that you have a

70
00:02:55,440 --> 00:02:57,040
camera with

71
00:02:57,040 --> 00:02:58,480
multiple lenses

72
00:02:58,480 --> 00:03:00,720
so it can see everything around around

73
00:03:00,720 --> 00:03:02,159
it at once

74
00:03:02,159 --> 00:03:03,360
so

75
00:03:03,360 --> 00:03:06,080
on the left you have uh

76
00:03:06,080 --> 00:03:09,360
a diy early camera basically consists of

77
00:03:09,360 --> 00:03:11,840
six gopro cameras

78
00:03:11,840 --> 00:03:14,239
formed shaped in a cube so every every

79
00:03:14,239 --> 00:03:16,959
lens is looking a different direction

80
00:03:16,959 --> 00:03:19,760
and on the right we have

81
00:03:19,760 --> 00:03:21,120
also

82
00:03:21,120 --> 00:03:24,480
somewhat dated samsung cameras with two

83
00:03:24,480 --> 00:03:26,000
fisheye lenses

84
00:03:26,000 --> 00:03:28,640
and um

85
00:03:29,120 --> 00:03:31,040
the one lens on the front sees the one

86
00:03:31,040 --> 00:03:32,480
half of the image and the other one sees

87
00:03:32,480 --> 00:03:33,599
the other half

88
00:03:33,599 --> 00:03:35,840
and actually they're a bit more than 180

89
00:03:35,840 --> 00:03:37,760
degrees so it's

90
00:03:37,760 --> 00:03:39,360
you may think that's not even possible

91
00:03:39,360 --> 00:03:41,680
but there are lenses that can

92
00:03:41,680 --> 00:03:44,720
i've seen up to 220 degrees so basically

93
00:03:44,720 --> 00:03:46,319
the camera is kind of looking behind

94
00:03:46,319 --> 00:03:47,760
itself a bit

95
00:03:47,760 --> 00:03:49,599
and this is important because if you

96
00:03:49,599 --> 00:03:51,280
only have two lenses you need some

97
00:03:51,280 --> 00:03:53,920
overlap between the two lenses

98
00:03:53,920 --> 00:03:56,319
there will be a stitching

99
00:03:56,319 --> 00:03:59,200
involved in this overlap area and it

100
00:03:59,200 --> 00:04:02,799
ideally it's completely invisible

101
00:04:02,799 --> 00:04:03,519
but

102
00:04:03,519 --> 00:04:06,480
because if you look at the um

103
00:04:06,480 --> 00:04:08,959
at the samsung camera on the right

104
00:04:08,959 --> 00:04:11,040
um obviously there are two sensors one

105
00:04:11,040 --> 00:04:12,400
looking in this way and one looking the

106
00:04:12,400 --> 00:04:14,799
other way and there's some space in

107
00:04:14,799 --> 00:04:17,440
between yeah so this is uh

108
00:04:17,440 --> 00:04:19,759
this causes a parallax error

109
00:04:19,759 --> 00:04:21,759
and this has to be hidden

110
00:04:21,759 --> 00:04:22,960
by

111
00:04:22,960 --> 00:04:24,320
clever stitching

112
00:04:24,320 --> 00:04:25,840
and also it means the larger the

113
00:04:25,840 --> 00:04:28,400
parallax

114
00:04:28,400 --> 00:04:30,080
the further away you have to be from the

115
00:04:30,080 --> 00:04:32,479
camera in the void in order to avoid

116
00:04:32,479 --> 00:04:34,800
stitching errors

117
00:04:34,800 --> 00:04:36,800
so

118
00:04:36,800 --> 00:04:38,800
for this type of camera the stitching

119
00:04:38,800 --> 00:04:40,800
arrow is pretty small

120
00:04:40,800 --> 00:04:41,759
and

121
00:04:41,759 --> 00:04:44,400
here for example we have some

122
00:04:44,400 --> 00:04:47,520
on the left there's a camera google jump

123
00:04:47,520 --> 00:04:49,520
with 16 gopros

124
00:04:49,520 --> 00:04:50,560
and

125
00:04:50,560 --> 00:04:53,600
it has a pretty large

126
00:04:53,919 --> 00:04:56,240
parallax error but it does has a

127
00:04:56,240 --> 00:04:59,280
different approach so it uses

128
00:04:59,280 --> 00:05:00,960
some

129
00:05:00,960 --> 00:05:02,080
advanced

130
00:05:02,080 --> 00:05:04,800
computer vision to create a 3d

131
00:05:04,800 --> 00:05:07,440
video out of it so um

132
00:05:07,440 --> 00:05:09,360
may not be such a huge problem with that

133
00:05:09,360 --> 00:05:10,560
type of camera

134
00:05:10,560 --> 00:05:12,800
and on the right you have a another pro

135
00:05:12,800 --> 00:05:15,120
camera with eight lenses

136
00:05:15,120 --> 00:05:16,240
and

137
00:05:16,240 --> 00:05:20,080
it can also generate a 3d video out of

138
00:05:20,080 --> 00:05:23,440
out of the individual lenses

139
00:05:23,440 --> 00:05:24,720
um

140
00:05:24,720 --> 00:05:26,400
today i'm using for the for the live

141
00:05:26,400 --> 00:05:28,479
stream i'm using the camera on the left

142
00:05:28,479 --> 00:05:30,479
and yes you can see it has eight lenses

143
00:05:30,479 --> 00:05:31,919
uh

144
00:05:31,919 --> 00:05:33,680
four times two looking in all four

145
00:05:33,680 --> 00:05:35,440
directions and they are wide angle

146
00:05:35,440 --> 00:05:38,800
lenses so they can look up and down also

147
00:05:38,800 --> 00:05:41,520
obviously the camera

148
00:05:41,520 --> 00:05:43,919
cannot see what's directly above or

149
00:05:43,919 --> 00:05:46,880
below it just a bit further away

150
00:05:46,880 --> 00:05:49,520
that's a

151
00:05:49,520 --> 00:05:51,199
common feature

152
00:05:51,199 --> 00:05:53,919
of these cameras for example if you

153
00:05:53,919 --> 00:05:55,520
if you have the camera on the left you

154
00:05:55,520 --> 00:05:57,759
can put it on the monopod and then if

155
00:05:57,759 --> 00:05:59,840
you're wondering how can i illuminate my

156
00:05:59,840 --> 00:06:01,520
scene uh

157
00:06:01,520 --> 00:06:03,520
if you don't want the the camera light

158
00:06:03,520 --> 00:06:04,960
to be in the in the picture everything

159
00:06:04,960 --> 00:06:07,120
is in the picture right so

160
00:06:07,120 --> 00:06:09,919
but by putting the um maybe an led strip

161
00:06:09,919 --> 00:06:11,600
on your on your monopod you can

162
00:06:11,600 --> 00:06:13,039
illuminate the scene and it will be

163
00:06:13,039 --> 00:06:15,440
invisible

164
00:06:15,440 --> 00:06:16,720
yeah

165
00:06:16,720 --> 00:06:18,840
okay so

166
00:06:18,840 --> 00:06:23,360
um these cameras deliver

167
00:06:23,360 --> 00:06:26,240
um the lenses are usually fisheye lenses

168
00:06:26,240 --> 00:06:29,440
and um we have to turn it into one

169
00:06:29,440 --> 00:06:31,440
coherent picture

170
00:06:31,440 --> 00:06:33,440
and you're probably familiar with this

171
00:06:33,440 --> 00:06:35,639
view of the globe that's called an

172
00:06:35,639 --> 00:06:38,720
equirectangular projection

173
00:06:38,720 --> 00:06:40,400
and

174
00:06:40,400 --> 00:06:42,240
basically it projects a sphere onto a

175
00:06:42,240 --> 00:06:43,440
plane

176
00:06:43,440 --> 00:06:44,560
and

177
00:06:44,560 --> 00:06:46,639
some people believe the earth is flat

178
00:06:46,639 --> 00:06:48,560
but i can assure you

179
00:06:48,560 --> 00:06:50,319
it's not

180
00:06:50,319 --> 00:06:51,840
and

181
00:06:51,840 --> 00:06:56,240
there this projection has a an error

182
00:06:56,240 --> 00:06:58,000
um

183
00:06:58,000 --> 00:07:00,319
oh yeah first something else

184
00:07:00,319 --> 00:07:03,280
so you can also do a 360 camera

185
00:07:03,280 --> 00:07:05,599
broadcast and just transmit a normal

186
00:07:05,599 --> 00:07:06,800
video by

187
00:07:06,800 --> 00:07:09,840
choosing a small part of the image and

188
00:07:09,840 --> 00:07:11,680
re it's called reframing

189
00:07:11,680 --> 00:07:13,360
for example you can have a camera on

190
00:07:13,360 --> 00:07:15,440
your table and you can

191
00:07:15,440 --> 00:07:17,280
some cameras offer

192
00:07:17,280 --> 00:07:19,759
tracking features so if there's a person

193
00:07:19,759 --> 00:07:21,599
in the image you can select it in your

194
00:07:21,599 --> 00:07:23,039
app and then it will track the person

195
00:07:23,039 --> 00:07:24,400
across a room

196
00:07:24,400 --> 00:07:27,280
and it's like a virtual camera movement

197
00:07:27,280 --> 00:07:29,440
but it's all done digitally and the

198
00:07:29,440 --> 00:07:31,759
result is just a normal image so this is

199
00:07:31,759 --> 00:07:34,080
not what this talk is about

200
00:07:34,080 --> 00:07:36,319
um

201
00:07:36,319 --> 00:07:39,039
yeah we want the viewer to be able to do

202
00:07:39,039 --> 00:07:40,319
the reframing

203
00:07:40,319 --> 00:07:42,400
himself or herself

204
00:07:42,400 --> 00:07:45,520
so um coming back to this projection

205
00:07:45,520 --> 00:07:48,879
here you can see uh the distortion

206
00:07:48,879 --> 00:07:51,440
the closer you get to the

207
00:07:51,440 --> 00:07:53,840
zenith and to the nettier

208
00:07:53,840 --> 00:07:56,479
um the larger the distortion is and it's

209
00:07:56,479 --> 00:07:58,800
basically infinite at this point because

210
00:07:58,800 --> 00:08:01,919
the poles would be like only a point

211
00:08:01,919 --> 00:08:04,240
and in this projection they go across

212
00:08:04,240 --> 00:08:06,240
the whole image

213
00:08:06,240 --> 00:08:08,800
and then it's the task of the the video

214
00:08:08,800 --> 00:08:09,840
viewer

215
00:08:09,840 --> 00:08:11,840
you need a special video viewer for 360

216
00:08:11,840 --> 00:08:13,280
video to

217
00:08:13,280 --> 00:08:17,919
um to undo this distortion yeah

218
00:08:17,919 --> 00:08:20,160
i will show you later um

219
00:08:20,160 --> 00:08:22,800
a little live demo if you haven't seen

220
00:08:22,800 --> 00:08:25,120
there was also the url on the first

221
00:08:25,120 --> 00:08:27,440
slide if you want to check it on your

222
00:08:27,440 --> 00:08:29,199
phone maybe i can go back to it real

223
00:08:29,199 --> 00:08:31,520
quick

224
00:08:32,559 --> 00:08:35,039
for what

225
00:08:37,039 --> 00:08:39,039
yeah you need some certain amount of

226
00:08:39,039 --> 00:08:40,880
bandwidth so if you all check it it will

227
00:08:40,880 --> 00:08:43,200
remember it might break actually so it's

228
00:08:43,200 --> 00:08:45,120
t one p dot d e

229
00:08:45,120 --> 00:08:47,200
m c h 20 20.

230
00:08:47,200 --> 00:08:49,440
okay

231
00:08:52,959 --> 00:08:55,440
okay so um

232
00:08:55,440 --> 00:08:57,920
let's talk about cameras some more about

233
00:08:57,920 --> 00:09:00,160
uh cameras that you can actually use for

234
00:09:00,160 --> 00:09:01,360
live streaming

235
00:09:01,360 --> 00:09:02,959
i mean there are

236
00:09:02,959 --> 00:09:04,320
there are quite a few cameras on the

237
00:09:04,320 --> 00:09:05,760
market these days

238
00:09:05,760 --> 00:09:06,640
but

239
00:09:06,640 --> 00:09:08,720
many of them have some severe

240
00:09:08,720 --> 00:09:11,040
restrictions

241
00:09:11,040 --> 00:09:12,720
so

242
00:09:12,720 --> 00:09:16,320
for example all insta 360 cameras

243
00:09:16,320 --> 00:09:18,800
that are non-pro cameras

244
00:09:18,800 --> 00:09:19,600
they

245
00:09:19,600 --> 00:09:23,279
cannot really stream in 4k and 4k is

246
00:09:23,279 --> 00:09:26,959
considered the minimum i would say

247
00:09:26,959 --> 00:09:30,240
i will talk a bit later about why y4k is

248
00:09:30,240 --> 00:09:32,399
the minimum requirement

249
00:09:32,399 --> 00:09:34,640
also the gopro max which is quite

250
00:09:34,640 --> 00:09:37,760
popular can only do full hd

251
00:09:37,760 --> 00:09:38,350
and

252
00:09:38,350 --> 00:09:39,519
[Music]

253
00:09:39,519 --> 00:09:42,320
the samsung gear 360 that i showed you

254
00:09:42,320 --> 00:09:44,720
which is quite a few years old it can do

255
00:09:44,720 --> 00:09:47,200
2k basically

256
00:09:47,200 --> 00:09:49,680
yeah so

257
00:09:49,680 --> 00:09:51,200
there are like

258
00:09:51,200 --> 00:09:52,880
three that i'm aware of of four if you

259
00:09:52,880 --> 00:09:55,279
count the voods and the vuvuz plus that

260
00:09:55,279 --> 00:09:57,200
can that cost less than one thousand

261
00:09:57,200 --> 00:10:00,000
euros and can do it

262
00:10:00,000 --> 00:10:02,959
uh the voos can also do 3d

263
00:10:02,959 --> 00:10:05,120
and then if you go to 2 000 euros you

264
00:10:05,120 --> 00:10:06,079
can

265
00:10:06,079 --> 00:10:09,279
those two cameras the lap panel and the

266
00:10:09,279 --> 00:10:12,480
q cam they can also do 8k streaming

267
00:10:12,480 --> 00:10:14,079
which obviously requires a lot of

268
00:10:14,079 --> 00:10:15,839
bandwidth

269
00:10:15,839 --> 00:10:18,839
um

270
00:10:19,279 --> 00:10:20,399
yeah so

271
00:10:20,399 --> 00:10:24,000
some some practical advice if you have a

272
00:10:24,000 --> 00:10:26,800
360 camera don't don't hold it directly

273
00:10:26,800 --> 00:10:28,320
in your hand because your hand will be

274
00:10:28,320 --> 00:10:30,399
so close to the camera it will look

275
00:10:30,399 --> 00:10:33,920
really creepy oh something i have to

276
00:10:33,920 --> 00:10:35,440
check i think something went wrong with

277
00:10:35,440 --> 00:10:37,839
the stream

278
00:10:38,560 --> 00:10:42,000
it's okay because it was beeping

279
00:10:44,399 --> 00:10:47,399
okay

280
00:10:47,839 --> 00:10:49,760
so don't hold it directly in your hand

281
00:10:49,760 --> 00:10:50,800
ideally

282
00:10:50,800 --> 00:10:53,279
you put it on a tripod because if people

283
00:10:53,279 --> 00:10:54,560
watch your

284
00:10:54,560 --> 00:10:56,560
video on a

285
00:10:56,560 --> 00:10:58,480
vr headset

286
00:10:58,480 --> 00:11:00,240
they might get sick if if it's shaking

287
00:11:00,240 --> 00:11:02,000
too much if you want to hold it in your

288
00:11:02,000 --> 00:11:03,920
hand use a gimbal

289
00:11:03,920 --> 00:11:05,120
and uh

290
00:11:05,120 --> 00:11:06,800
make sure you observe the minimum

291
00:11:06,800 --> 00:11:07,920
distance

292
00:11:07,920 --> 00:11:09,440
if your

293
00:11:09,440 --> 00:11:11,279
this camera also has a quite large

294
00:11:11,279 --> 00:11:12,640
minimum distance

295
00:11:12,640 --> 00:11:15,839
but if you make sure that

296
00:11:15,839 --> 00:11:17,279
that you are

297
00:11:17,279 --> 00:11:19,440
not on top of one of the stitching lines

298
00:11:19,440 --> 00:11:20,880
between the lenses

299
00:11:20,880 --> 00:11:24,880
then you can get away with being closer

300
00:11:24,880 --> 00:11:27,680
than the minimum distance without being

301
00:11:27,680 --> 00:11:29,600
cut off

302
00:11:29,600 --> 00:11:31,519
yeah so um

303
00:11:31,519 --> 00:11:34,240
we want a resolution of 4k

304
00:11:34,240 --> 00:11:35,519
why 4k

305
00:11:35,519 --> 00:11:36,560
because

306
00:11:36,560 --> 00:11:37,519
um

307
00:11:37,519 --> 00:11:41,519
the 4000 horizontal pixels are 3840 or

308
00:11:41,519 --> 00:11:43,920
whatever they are spread out across

309
00:11:43,920 --> 00:11:45,760
360 degrees

310
00:11:45,760 --> 00:11:48,560
and normally we are used to maybe 90

311
00:11:48,560 --> 00:11:49,600
degrees

312
00:11:49,600 --> 00:11:50,959
or even less

313
00:11:50,959 --> 00:11:53,600
depending on the kind of lens you have

314
00:11:53,600 --> 00:11:56,320
so if you if you compare a 360 camera

315
00:11:56,320 --> 00:11:59,200
with 4k to a regular camera

316
00:11:59,200 --> 00:12:02,480
then you quickly see that it's um

317
00:12:02,480 --> 00:12:04,880
after reframing it will be much much

318
00:12:04,880 --> 00:12:06,480
lower resolution maybe it will be look

319
00:12:06,480 --> 00:12:08,560
like 720p

320
00:12:08,560 --> 00:12:09,680
which is

321
00:12:09,680 --> 00:12:12,079
which is not so great but you know it's

322
00:12:12,079 --> 00:12:13,839
acceptable

323
00:12:13,839 --> 00:12:14,800
um

324
00:12:14,800 --> 00:12:18,399
yeah so 4k is quite important

325
00:12:18,399 --> 00:12:20,480
we use rtmp

326
00:12:20,480 --> 00:12:23,839
because that's what most cameras

327
00:12:23,839 --> 00:12:25,120
and

328
00:12:25,120 --> 00:12:26,639
the bit rate

329
00:12:26,639 --> 00:12:29,040
yeah mostly depends on what's what

330
00:12:29,040 --> 00:12:30,560
bandwidth you have available if you are

331
00:12:30,560 --> 00:12:33,040
somewhere on the go

332
00:12:33,040 --> 00:12:34,959
here i managed to get an ethernet cable

333
00:12:34,959 --> 00:12:35,839
so

334
00:12:35,839 --> 00:12:38,399
with wi-fi it's it's quite quite quite

335
00:12:38,399 --> 00:12:40,160
bad

336
00:12:40,160 --> 00:12:43,440
so here's a screenshot of the uh

337
00:12:43,440 --> 00:12:46,160
of the wu software running on a pc

338
00:12:46,160 --> 00:12:48,399
it's quite nice it has a lot of features

339
00:12:48,399 --> 00:12:50,560
but basically what you always have is

340
00:12:50,560 --> 00:12:52,560
you can

341
00:12:52,560 --> 00:12:56,480
enter your rtmp server address

342
00:12:56,480 --> 00:12:58,560
sometimes you yeah most of the time you

343
00:12:58,560 --> 00:12:59,839
can

344
00:12:59,839 --> 00:13:01,920
choose your resolution

345
00:13:01,920 --> 00:13:04,000
sometimes the bit rate

346
00:13:04,000 --> 00:13:04,839
and

347
00:13:04,839 --> 00:13:08,399
fps if your camera allows that and in

348
00:13:08,399 --> 00:13:10,639
this case we can also toggle between 2d

349
00:13:10,639 --> 00:13:12,480
and 3d

350
00:13:12,480 --> 00:13:14,399
and for those of you

351
00:13:14,399 --> 00:13:15,839
who are doing the token challenge

352
00:13:15,839 --> 00:13:19,279
there's also a token on this slide

353
00:13:19,279 --> 00:13:23,600
um okay so let's talk about rtmp

354
00:13:23,600 --> 00:13:26,880
rtmp is a real-time messaging protocol

355
00:13:26,880 --> 00:13:29,760
it's a proprietary protocol developed by

356
00:13:29,760 --> 00:13:33,200
macromedia back in the days of flash

357
00:13:33,200 --> 00:13:35,040
to communicate between the streaming

358
00:13:35,040 --> 00:13:38,639
server and the flash running on the pc

359
00:13:38,639 --> 00:13:41,199
and so it's proprietary so why is

360
00:13:41,199 --> 00:13:44,000
everyone using it well after

361
00:13:44,000 --> 00:13:47,279
adobe bought macromedia they published

362
00:13:47,279 --> 00:13:50,959
an incomplete spec for public use

363
00:13:50,959 --> 00:13:53,600
it uses tcp port

364
00:13:53,600 --> 00:13:56,240
1935

365
00:13:56,240 --> 00:13:58,480
and

366
00:13:58,480 --> 00:14:00,240
which is sometimes blocked in firewalls

367
00:14:00,240 --> 00:14:03,519
these days but

368
00:14:03,519 --> 00:14:05,920
for the first mile delivery which is the

369
00:14:05,920 --> 00:14:07,680
the from the camera to the streaming

370
00:14:07,680 --> 00:14:09,600
server it's it's a protocol of choice

371
00:14:09,600 --> 00:14:11,279
for sure

372
00:14:11,279 --> 00:14:13,600
the last mile delivery which used to be

373
00:14:13,600 --> 00:14:17,040
your browser running flash

374
00:14:17,199 --> 00:14:21,279
browsers can no longer play rtmp

375
00:14:21,360 --> 00:14:24,880
i mean you could use something like vlc

376
00:14:24,880 --> 00:14:27,600
to view an rtmp stream

377
00:14:27,600 --> 00:14:30,480
but vlc has some particular issues that

378
00:14:30,480 --> 00:14:32,560
it often doesn't recognize the video as

379
00:14:32,560 --> 00:14:34,160
being 360 and it

380
00:14:34,160 --> 00:14:36,639
offers no manual override so you cannot

381
00:14:36,639 --> 00:14:39,279
say this is a 360 video please distort

382
00:14:39,279 --> 00:14:42,959
it i mean undistorted

383
00:14:42,959 --> 00:14:44,639
so

384
00:14:44,639 --> 00:14:48,399
that's uh really a bummer

385
00:14:48,399 --> 00:14:50,560
and yeah there are also some proprietary

386
00:14:50,560 --> 00:14:53,040
apps on different platforms that can do

387
00:14:53,040 --> 00:14:56,639
rtmp but today i want to

388
00:14:56,639 --> 00:14:58,800
talk about how to do the streaming

389
00:14:58,800 --> 00:15:00,959
without proprietary plugins so

390
00:15:00,959 --> 00:15:03,680
we're using hls which is http live

391
00:15:03,680 --> 00:15:06,160
streaming

392
00:15:06,160 --> 00:15:08,399
which was pretty much invented to fill

393
00:15:08,399 --> 00:15:10,480
the gap that uh

394
00:15:10,480 --> 00:15:11,199
that

395
00:15:11,199 --> 00:15:14,079
the disappearance of flash left behind

396
00:15:14,079 --> 00:15:15,199
and

397
00:15:15,199 --> 00:15:18,000
which it's funny because uh hls was

398
00:15:18,000 --> 00:15:20,240
actually invented by apple so

399
00:15:20,240 --> 00:15:22,399
they killed flash so they i guess they

400
00:15:22,399 --> 00:15:27,040
had to uh come up with something else

401
00:15:27,040 --> 00:15:29,759
so hls

402
00:15:29,759 --> 00:15:32,000
has

403
00:15:33,600 --> 00:15:36,320
chops up the video in small

404
00:15:36,320 --> 00:15:38,959
small smaller files and they are just

405
00:15:38,959 --> 00:15:44,319
downloaded normally using http or https

406
00:15:45,120 --> 00:15:48,240
so there is a problem with latency so it

407
00:15:48,240 --> 00:15:49,920
can can be

408
00:15:49,920 --> 00:15:52,959
between 10 20 30 seconds sometimes as

409
00:15:52,959 --> 00:15:54,720
much as one minute

410
00:15:54,720 --> 00:15:57,680
there's also a newer variant called low

411
00:15:57,680 --> 00:16:00,079
latency hls

412
00:16:00,079 --> 00:16:01,440
and

413
00:16:01,440 --> 00:16:04,079
some browsers have some

414
00:16:04,079 --> 00:16:06,399
restrictions

415
00:16:06,399 --> 00:16:08,160
with

416
00:16:08,160 --> 00:16:09,759
with hls

417
00:16:09,759 --> 00:16:13,440
in terms of uh requiring https

418
00:16:13,440 --> 00:16:14,560
so

419
00:16:14,560 --> 00:16:16,560
to avoid it all just

420
00:16:16,560 --> 00:16:18,560
get a certificate right away and don't

421
00:16:18,560 --> 00:16:19,920
bother

422
00:16:19,920 --> 00:16:24,399
setting it up without https

423
00:16:27,600 --> 00:16:28,800
yeah

424
00:16:28,800 --> 00:16:31,040
okay

425
00:16:31,120 --> 00:16:32,000
so

426
00:16:32,000 --> 00:16:33,920
what we want to do is we want to stream

427
00:16:33,920 --> 00:16:37,360
the camera image via rtmp to an rtmp

428
00:16:37,360 --> 00:16:40,160
server and the rtmp server will provide

429
00:16:40,160 --> 00:16:41,920
hls

430
00:16:41,920 --> 00:16:44,639
to which can then be streamed to the

431
00:16:44,639 --> 00:16:46,720
browser

432
00:16:46,720 --> 00:16:48,079
and

433
00:16:48,079 --> 00:16:50,399
in terms of free software

434
00:16:50,399 --> 00:16:52,560
there are four that i'm aware of

435
00:16:52,560 --> 00:16:54,240
there's nginx

436
00:16:54,240 --> 00:16:56,240
with the rtmp module

437
00:16:56,240 --> 00:16:59,759
which is by far the most popular option

438
00:16:59,759 --> 00:17:03,120
and it's also easy to set up

439
00:17:03,120 --> 00:17:06,160
uh it's what i'm using today

440
00:17:06,160 --> 00:17:09,520
it has some some drawbacks um

441
00:17:09,520 --> 00:17:11,359
there hasn't been any development really

442
00:17:11,359 --> 00:17:14,400
since since 2017

443
00:17:14,400 --> 00:17:17,039
and so that means there's no low latency

444
00:17:17,039 --> 00:17:19,839
hls support

445
00:17:22,400 --> 00:17:24,799
and media server they they have a dual

446
00:17:24,799 --> 00:17:26,959
license server so they have a lot of

447
00:17:26,959 --> 00:17:29,360
enterprise features which is which are

448
00:17:29,360 --> 00:17:31,360
only available in the in the

449
00:17:31,360 --> 00:17:33,760
expensive version

450
00:17:33,760 --> 00:17:35,520
something i don't really like that much

451
00:17:35,520 --> 00:17:37,760
because usually

452
00:17:37,760 --> 00:17:39,760
sooner or later the free version ends up

453
00:17:39,760 --> 00:17:43,360
being useless with this kind of setup

454
00:17:43,360 --> 00:17:46,080
yeah oven media looks looks really

455
00:17:46,080 --> 00:17:48,320
interesting but i kind of ran out of

456
00:17:48,320 --> 00:17:51,679
time otherwise i would have tried

457
00:17:51,679 --> 00:17:53,280
setting up that one because it also

458
00:17:53,280 --> 00:17:56,720
supports low latency hls

459
00:17:56,720 --> 00:17:59,120
okay so

460
00:17:59,120 --> 00:18:02,080
nginx super easy to set up

461
00:18:02,080 --> 00:18:05,120
basically install nginx then install the

462
00:18:05,120 --> 00:18:07,919
mod on with you have like a debian based

463
00:18:07,919 --> 00:18:09,840
linux

464
00:18:09,840 --> 00:18:12,720
that's the command to install it

465
00:18:12,720 --> 00:18:15,919
then make sure you have ssl running

466
00:18:15,919 --> 00:18:18,160
with certbot for example

467
00:18:18,160 --> 00:18:19,360
and then

468
00:18:19,360 --> 00:18:21,039
this is basically

469
00:18:21,039 --> 00:18:22,400
um

470
00:18:22,400 --> 00:18:24,160
your minimum

471
00:18:24,160 --> 00:18:26,799
nginx configuration file yeah so you

472
00:18:26,799 --> 00:18:29,280
have a block inside the file

473
00:18:29,280 --> 00:18:32,160
without for the rtmp module

474
00:18:32,160 --> 00:18:35,200
and then you can define an application

475
00:18:35,200 --> 00:18:36,880
and the name of the application is

476
00:18:36,880 --> 00:18:39,840
actually the the url

477
00:18:39,840 --> 00:18:41,280
so

478
00:18:41,280 --> 00:18:43,840
it will be slash app something

479
00:18:43,840 --> 00:18:45,360
then you turn on live

480
00:18:45,360 --> 00:18:47,840
you can also record

481
00:18:47,840 --> 00:18:50,720
using nginx if you want to have

482
00:18:50,720 --> 00:18:52,480
a recording of your stream which is

483
00:18:52,480 --> 00:18:54,400
quite nice i think

484
00:18:54,400 --> 00:18:57,120
um

485
00:18:57,120 --> 00:18:58,400
yeah and then

486
00:18:58,400 --> 00:19:00,480
using the hls option

487
00:19:00,480 --> 00:19:02,880
you define it to

488
00:19:02,880 --> 00:19:05,200
i mean you configure it to provide an

489
00:19:05,200 --> 00:19:06,720
hls stream

490
00:19:06,720 --> 00:19:09,440
at this point you can also

491
00:19:09,440 --> 00:19:11,919
restrict the rtmp stream if you if you

492
00:19:11,919 --> 00:19:14,320
want to force your users to to use the

493
00:19:14,320 --> 00:19:16,559
hls stream for some reason

494
00:19:16,559 --> 00:19:17,919
and you can try fiddling with the

495
00:19:17,919 --> 00:19:19,520
fragment size

496
00:19:19,520 --> 00:19:22,160
to reduce latency but usually

497
00:19:22,160 --> 00:19:23,840
if you

498
00:19:23,840 --> 00:19:25,440
get it too small then

499
00:19:25,440 --> 00:19:27,840
you get stuttering or

500
00:19:27,840 --> 00:19:30,640
quality suffers

501
00:19:30,640 --> 00:19:32,240
okay

502
00:19:32,240 --> 00:19:33,520
now we have

503
00:19:33,520 --> 00:19:35,440
the rtmp stream going into the server

504
00:19:35,440 --> 00:19:37,840
and we have hls from the server

505
00:19:37,840 --> 00:19:39,840
available so how can we get it to play

506
00:19:39,840 --> 00:19:41,679
in the browser

507
00:19:41,679 --> 00:19:44,320
as i said hls is

508
00:19:44,320 --> 00:19:46,480
in theory it's supported

509
00:19:46,480 --> 00:19:48,960
by browsers more recent browsers

510
00:19:48,960 --> 00:19:52,720
but in practice only safari supports it

511
00:19:52,720 --> 00:19:55,440
directly in the video tag

512
00:19:55,440 --> 00:19:57,360
but there are some

513
00:19:57,360 --> 00:19:59,360
various javascript libraries available

514
00:19:59,360 --> 00:20:01,760
that will

515
00:20:01,919 --> 00:20:03,600
make sure it works

516
00:20:03,600 --> 00:20:04,960
by

517
00:20:04,960 --> 00:20:07,840
using all kinds of different

518
00:20:07,840 --> 00:20:11,039
technologies under the hood

519
00:20:11,440 --> 00:20:14,159
there's some multimedia extension

520
00:20:14,159 --> 00:20:17,039
which is available on most browsers

521
00:20:17,039 --> 00:20:18,840
and hls uses

522
00:20:18,840 --> 00:20:21,280
it but for example

523
00:20:21,280 --> 00:20:23,280
on ios that's

524
00:20:23,280 --> 00:20:25,679
missing again so

525
00:20:25,679 --> 00:20:29,120
if you just use hls js by itself it will

526
00:20:29,120 --> 00:20:32,000
not work on ios

527
00:20:32,000 --> 00:20:35,360
yeah and then you have a video playing

528
00:20:35,360 --> 00:20:38,159
in your browser but remember we have an

529
00:20:38,159 --> 00:20:40,559
acrylic rectangular image so it will

530
00:20:40,559 --> 00:20:42,880
look like crap

531
00:20:42,880 --> 00:20:46,640
so you also need um

532
00:20:46,960 --> 00:20:52,559
to to reframe the video to the 360 video

533
00:20:52,559 --> 00:20:55,520
and let the users

534
00:20:55,520 --> 00:20:58,320
pick what they're looking at

535
00:20:58,320 --> 00:21:00,559
usually this is done with if you're

536
00:21:00,559 --> 00:21:02,080
looking on a desktop you can click on

537
00:21:02,080 --> 00:21:03,760
the video and

538
00:21:03,760 --> 00:21:06,000
drag with your mouse to look around in

539
00:21:06,000 --> 00:21:08,720
in the sphere basically or if you have a

540
00:21:08,720 --> 00:21:10,720
touch screen device you can just

541
00:21:10,720 --> 00:21:12,799
drag with your finger

542
00:21:12,799 --> 00:21:15,919
and if you have a device that has an imu

543
00:21:15,919 --> 00:21:19,440
like a phone a positional sensor you can

544
00:21:19,440 --> 00:21:21,600
also hold it in front of you and look

545
00:21:21,600 --> 00:21:25,039
around in in your image

546
00:21:25,039 --> 00:21:27,679
which is kind of nice and the vr goggles

547
00:21:27,679 --> 00:21:30,799
also use imu so it's like standing

548
00:21:30,799 --> 00:21:32,240
inside

549
00:21:32,240 --> 00:21:34,240
standing where the camera is standing

550
00:21:34,240 --> 00:21:36,799
and you're looking around in the image

551
00:21:36,799 --> 00:21:38,080
so

552
00:21:38,080 --> 00:21:40,799
kaleidoscope is one of those 360 video

553
00:21:40,799 --> 00:21:42,320
viewers

554
00:21:42,320 --> 00:21:45,600
it's it's quite simple and it lacking a

555
00:21:45,600 --> 00:21:48,720
few features in particular um

556
00:21:48,720 --> 00:21:50,880
yeah it doesn't support

557
00:21:50,880 --> 00:21:52,960
vr goggles and so on

558
00:21:52,960 --> 00:21:56,559
so another one i found was eg js

559
00:21:56,559 --> 00:21:59,520
and that's what i ended up using

560
00:21:59,520 --> 00:22:01,200
egs is a

561
00:22:01,200 --> 00:22:03,280
java component group

562
00:22:03,280 --> 00:22:04,159
and

563
00:22:04,159 --> 00:22:07,280
one of the components is called vue 360

564
00:22:07,280 --> 00:22:08,880
and it has

565
00:22:08,880 --> 00:22:10,840
a sub component called

566
00:22:10,840 --> 00:22:13,520
panoviewer which is a panoramic media

567
00:22:13,520 --> 00:22:14,799
viewer

568
00:22:14,799 --> 00:22:16,720
and it supports

569
00:22:16,720 --> 00:22:19,919
tons of projections aquary rectangular

570
00:22:19,919 --> 00:22:22,240
cube maps which is if you what you have

571
00:22:22,240 --> 00:22:23,919
if you have like

572
00:22:23,919 --> 00:22:26,080
remember the the gopro camera i showed

573
00:22:26,080 --> 00:22:27,679
you in the beginning with six cameras

574
00:22:27,679 --> 00:22:29,120
shaped like a cube

575
00:22:29,120 --> 00:22:30,400
and

576
00:22:30,400 --> 00:22:33,120
so basically you have six videos

577
00:22:33,120 --> 00:22:36,960
and uh this viewer can

578
00:22:36,960 --> 00:22:38,400
arrange it so

579
00:22:38,400 --> 00:22:40,559
it works seamlessly

580
00:22:40,559 --> 00:22:43,120
and it also supports stereoscopy

581
00:22:43,120 --> 00:22:45,440
rectangular equirectangular

582
00:22:45,440 --> 00:22:48,880
images which is just saying

583
00:22:48,880 --> 00:22:51,760
so you have an acrylic rectangular image

584
00:22:51,760 --> 00:22:54,080
and in the case of this camera you have

585
00:22:54,080 --> 00:22:56,080
actually two of them

586
00:22:56,080 --> 00:22:57,200
and uh

587
00:22:57,200 --> 00:22:59,280
one is always the left eye the left

588
00:22:59,280 --> 00:23:01,039
camera of the two and the other one is

589
00:23:01,039 --> 00:23:02,320
over the right

590
00:23:02,320 --> 00:23:04,880
and then this viewer supports it so if

591
00:23:04,880 --> 00:23:07,120
if you watch it

592
00:23:07,120 --> 00:23:10,240
it will actually be stereo

593
00:23:10,240 --> 00:23:13,360
it runs on browsers with stable webgl

594
00:23:13,360 --> 00:23:15,360
which means it runs on chrome firefox

595
00:23:15,360 --> 00:23:19,360
safari edge even internet explorer l11

596
00:23:19,360 --> 00:23:22,559
and on mobile it works on mobile safari

597
00:23:22,559 --> 00:23:25,280
not just on ipad os but also on ios it

598
00:23:25,280 --> 00:23:27,280
works with chrome on android

599
00:23:27,280 --> 00:23:30,400
and it works with a samsung internet

600
00:23:30,400 --> 00:23:33,039
browser which is what's running on

601
00:23:33,039 --> 00:23:36,000
oculus go for example

602
00:23:36,000 --> 00:23:38,000
yeah and it uses under the hood it uses

603
00:23:38,000 --> 00:23:42,880
hls.js to play the to play the stream

604
00:23:42,880 --> 00:23:43,919
um

605
00:23:43,919 --> 00:23:46,000
yeah so some more

606
00:23:46,000 --> 00:23:48,159
features that can run full screen you

607
00:23:48,159 --> 00:23:50,720
can change orientation with the mouse

608
00:23:50,720 --> 00:23:51,600
um

609
00:23:51,600 --> 00:23:53,679
and one particular feature which i think

610
00:23:53,679 --> 00:23:56,320
is really cool um if you

611
00:23:56,320 --> 00:23:58,080
if you have a phone

612
00:23:58,080 --> 00:23:59,760
maybe you remember the old google

613
00:23:59,760 --> 00:24:01,360
cardboard

614
00:24:01,360 --> 00:24:05,158
so basically um

615
00:24:05,200 --> 00:24:06,720
cardboard is

616
00:24:06,720 --> 00:24:08,559
having two lenses in front of your phone

617
00:24:08,559 --> 00:24:09,600
display

618
00:24:09,600 --> 00:24:10,799
and then

619
00:24:10,799 --> 00:24:14,000
having the phone display

620
00:24:14,000 --> 00:24:15,919
divided into two halves

621
00:24:15,919 --> 00:24:18,559
and you have two lenses and um

622
00:24:18,559 --> 00:24:21,039
every eye sees something else

623
00:24:21,039 --> 00:24:25,760
and um so which means it's it's stereo

624
00:24:26,480 --> 00:24:29,679
and uh the panel viewer supports this so

625
00:24:29,679 --> 00:24:32,960
yeah that's that's quite nice so let's

626
00:24:32,960 --> 00:24:35,840
let's check it out

627
00:24:36,159 --> 00:24:39,760
i'm switching hopefully

628
00:24:39,760 --> 00:24:41,279
to the

629
00:24:41,279 --> 00:24:42,960
browser here

630
00:24:42,960 --> 00:24:44,799
yeah let's see if the stream is still

631
00:24:44,799 --> 00:24:47,120
running looks good

632
00:24:47,120 --> 00:24:49,520
so now

633
00:24:49,520 --> 00:24:52,559
i can look around here

634
00:24:54,480 --> 00:24:57,919
maybe i can show you some things

635
00:24:57,919 --> 00:24:59,039
you will see

636
00:24:59,039 --> 00:25:01,360
for example here this this line this is

637
00:25:01,360 --> 00:25:03,200
the stitching line

638
00:25:03,200 --> 00:25:05,679
which is very obvious because the table

639
00:25:05,679 --> 00:25:07,919
is too close to the camera here ideally

640
00:25:07,919 --> 00:25:10,799
we would have a higher tripod

641
00:25:10,799 --> 00:25:13,200
and the stitching line is also apparent

642
00:25:13,200 --> 00:25:16,799
here somewhat in the

643
00:25:18,320 --> 00:25:21,200
in the mch logo

644
00:25:21,200 --> 00:25:22,799
yeah and you can also

645
00:25:22,799 --> 00:25:24,799
zoom out a bit and zoom in

646
00:25:24,799 --> 00:25:28,320
and it supports full screen

647
00:25:28,320 --> 00:25:29,200
so

648
00:25:29,200 --> 00:25:31,279
this is a really nice viewer

649
00:25:31,279 --> 00:25:33,039
and if you're running it on a phone you

650
00:25:33,039 --> 00:25:34,960
will get an extra icon in the top right

651
00:25:34,960 --> 00:25:36,559
corner for the

652
00:25:36,559 --> 00:25:38,559
cardboard mode

653
00:25:38,559 --> 00:25:39,840
and if you're running it on a virtual

654
00:25:39,840 --> 00:25:42,559
reality device such a oculus go it will

655
00:25:42,559 --> 00:25:44,400
also work

656
00:25:44,400 --> 00:25:46,640
so

657
00:25:47,520 --> 00:25:51,679
pretty pretty neat i'm totally

658
00:25:51,840 --> 00:25:54,000
oh

659
00:25:54,000 --> 00:25:55,120
something is wrong with the stream

660
00:25:55,120 --> 00:25:57,840
obviously

661
00:25:57,840 --> 00:26:00,159
because it's not this hasn't happened in

662
00:26:00,159 --> 00:26:01,600
a while i mean

663
00:26:01,600 --> 00:26:04,960
this person is not on the on the stage

664
00:26:04,960 --> 00:26:08,080
i have to check anyway

665
00:26:08,960 --> 00:26:10,720
what

666
00:26:10,720 --> 00:26:13,760
yeah i will restart it later

667
00:26:13,760 --> 00:26:16,320
anyway um

668
00:26:16,320 --> 00:26:20,080
let's go back to the presentation

669
00:26:20,720 --> 00:26:23,200
so some possible possible improvements

670
00:26:23,200 --> 00:26:26,720
besides the fact that the stream stopped

671
00:26:26,720 --> 00:26:28,960
is to switch the rtmp server to get low

672
00:26:28,960 --> 00:26:31,279
latency hls

673
00:26:31,279 --> 00:26:33,520
and there's also depending on your

674
00:26:33,520 --> 00:26:34,559
audience

675
00:26:34,559 --> 00:26:36,159
if you have a small

676
00:26:36,159 --> 00:26:37,760
audience

677
00:26:37,760 --> 00:26:41,919
you can also use a webrtc instead of hls

678
00:26:41,919 --> 00:26:45,120
which means a lot lower latency

679
00:26:45,120 --> 00:26:47,679
less than one second mostly

680
00:26:47,679 --> 00:26:49,679
but it doesn't really scale but if you

681
00:26:49,679 --> 00:26:52,559
have like a small number of

682
00:26:52,559 --> 00:26:54,080
clients maybe like

683
00:26:54,080 --> 00:26:56,000
tens of clients

684
00:26:56,000 --> 00:26:58,720
then webrtc is definitely a very

685
00:26:58,720 --> 00:27:00,159
interesting option

686
00:27:00,159 --> 00:27:02,320
and i saw that the oven media server

687
00:27:02,320 --> 00:27:03,600
also

688
00:27:03,600 --> 00:27:05,679
supports webrtc

689
00:27:05,679 --> 00:27:07,200
yeah and you can also do some

690
00:27:07,200 --> 00:27:10,159
interesting stuff on your rtmp server

691
00:27:10,159 --> 00:27:12,320
switching between multiple cameras you

692
00:27:12,320 --> 00:27:13,600
can

693
00:27:13,600 --> 00:27:16,960
add an overlay using ffmpeg into the 360

694
00:27:16,960 --> 00:27:19,120
image

695
00:27:19,120 --> 00:27:21,760
and

696
00:27:21,760 --> 00:27:24,399
yeah those are the two main improvements

697
00:27:24,399 --> 00:27:26,879
i would say

698
00:27:27,200 --> 00:27:28,399
okay

699
00:27:28,399 --> 00:27:29,760
that's it

700
00:27:29,760 --> 00:27:32,399
for the talk um

701
00:27:32,399 --> 00:27:35,840
do you have any questions

702
00:27:44,960 --> 00:27:47,360
okay if not we i will be around here so

703
00:27:47,360 --> 00:27:50,399
you can just get in touch after

704
00:27:50,399 --> 00:27:52,240
afterwards if you have any questions you

705
00:27:52,240 --> 00:27:53,520
can line up in the microphones in the

706
00:27:53,520 --> 00:27:55,120
middle

707
00:27:55,120 --> 00:27:57,279
and i'm not sure yeah i'm and do we have

708
00:27:57,279 --> 00:27:59,120
any questions on the internet

709
00:27:59,120 --> 00:28:01,120
also not

710
00:28:01,120 --> 00:28:02,960
okay

711
00:28:02,960 --> 00:28:06,000
so if no one wants to ask a question

712
00:28:06,000 --> 00:28:08,880
he will be around yes

713
00:28:08,880 --> 00:28:10,480
please give him a round of applause oh

714
00:28:10,480 --> 00:28:12,320
no there is a question maybe you have

715
00:28:12,320 --> 00:28:13,360
one

716
00:28:13,360 --> 00:28:15,679
then just come to the microphone please

717
00:28:15,679 --> 00:28:18,080
close close close hello can you hear me

718
00:28:18,080 --> 00:28:20,399
a while yes awesome um i was very

719
00:28:20,399 --> 00:28:21,600
curious for this presentation thank you

720
00:28:21,600 --> 00:28:23,279
so much for preparing this material and

721
00:28:23,279 --> 00:28:25,039
presenting i have kind of like a hobby

722
00:28:25,039 --> 00:28:26,000
project

723
00:28:26,000 --> 00:28:28,559
um which is just like it's about like

724
00:28:28,559 --> 00:28:31,120
showing outdoor adventures and so i'm

725
00:28:31,120 --> 00:28:32,799
very curious to use

726
00:28:32,799 --> 00:28:35,600
this tech egs and so on not just to

727
00:28:35,600 --> 00:28:38,880
stream like a 360 video but to stream

728
00:28:38,880 --> 00:28:41,440
multiple

729
00:28:41,440 --> 00:28:43,760
multiple video streams into one and like

730
00:28:43,760 --> 00:28:45,840
enabling people that are remote to to

731
00:28:45,840 --> 00:28:48,320
have uh to have a multiple views into

732
00:28:48,320 --> 00:28:51,200
what is happening uh so like uh i don't

733
00:28:51,200 --> 00:28:52,880
know maybe an analogy is thinking think

734
00:28:52,880 --> 00:28:53,679
of like

735
00:28:53,679 --> 00:28:58,480
those multiple screens on a nasa launch

736
00:28:58,480 --> 00:29:00,559
of the rocket like yeah yeah what is the

737
00:29:00,559 --> 00:29:03,200
best way to like bring multiple streams

738
00:29:03,200 --> 00:29:05,919
of video 360 or 360

739
00:29:05,919 --> 00:29:06,799
and give

740
00:29:06,799 --> 00:29:08,240
people the the opportunity to

741
00:29:08,240 --> 00:29:10,159
participate as they want

742
00:29:10,159 --> 00:29:13,200
yeah so the eg js is basically also an

743
00:29:13,200 --> 00:29:14,640
api so

744
00:29:14,640 --> 00:29:16,159
you would have to write a little bit of

745
00:29:16,159 --> 00:29:18,640
javascript to add maybe a button or

746
00:29:18,640 --> 00:29:20,880
something to switch between different

747
00:29:20,880 --> 00:29:21,919
sources

748
00:29:21,919 --> 00:29:25,760
but it will also be not live right so um

749
00:29:25,760 --> 00:29:27,279
uh so

750
00:29:27,279 --> 00:29:29,279
can't it be live or could you no i mean

751
00:29:29,279 --> 00:29:32,080
it could be live but i'm saying uh

752
00:29:32,080 --> 00:29:34,320
you you cannot so you will have maybe

753
00:29:34,320 --> 00:29:35,360
you will have

754
00:29:35,360 --> 00:29:37,039
either you will have multiple live

755
00:29:37,039 --> 00:29:38,880
cameras or you will have

756
00:29:38,880 --> 00:29:41,600
pre-recorded videos and you can allow

757
00:29:41,600 --> 00:29:43,600
your viewers to switch between them

758
00:29:43,600 --> 00:29:44,880
yeah but you would have to do a little

759
00:29:44,880 --> 00:29:47,360
bit of programming for sure yes

760
00:29:47,360 --> 00:29:50,080
out of the box all right no but it's not

761
00:29:50,080 --> 00:29:51,200
i mean

762
00:29:51,200 --> 00:29:53,039
i think it would be pretty easy actually

763
00:29:53,039 --> 00:29:55,120
awesome yeah all right thank you very

764
00:29:55,120 --> 00:29:57,120
welcome

765
00:29:57,120 --> 00:29:59,200
next question please yes another

766
00:29:59,200 --> 00:30:01,279
question um

767
00:30:01,279 --> 00:30:05,039
moreover remark the engine xrtmp

768
00:30:05,039 --> 00:30:08,080
module was kind of discontinued and

769
00:30:08,080 --> 00:30:10,880
another group forked it and i think they

770
00:30:10,880 --> 00:30:13,200
renamed it to engine xts and it has a

771
00:30:13,200 --> 00:30:15,360
few more features

772
00:30:15,360 --> 00:30:17,679
and also the current replacement for

773
00:30:17,679 --> 00:30:20,399
rtmp is srt

774
00:30:20,399 --> 00:30:24,559
are there 360 cameras supporting srt yet

775
00:30:24,559 --> 00:30:28,640
or ndi for local streaming

776
00:30:28,640 --> 00:30:31,520
well um ndi for sure but those are

777
00:30:31,520 --> 00:30:35,120
usually pro cameras uh in the 5000 plus

778
00:30:35,120 --> 00:30:37,840
euro range

779
00:30:37,919 --> 00:30:41,039
srt i don't know i'm not familiar with

780
00:30:41,039 --> 00:30:42,880
well

781
00:30:42,880 --> 00:30:45,679
but of course you can usually also use

782
00:30:45,679 --> 00:30:48,320
rtmps if you want to encrypt your stream

783
00:30:48,320 --> 00:30:50,320
to the rtmp server that should be

784
00:30:50,320 --> 00:30:53,840
supported by most cameras already

785
00:30:54,480 --> 00:30:56,399
firefox is required

786
00:30:56,399 --> 00:30:59,440
facebook is requiring it by now yeah

787
00:30:59,440 --> 00:31:01,600
no

788
00:31:01,760 --> 00:31:02,720
okay

789
00:31:02,720 --> 00:31:03,600
so

790
00:31:03,600 --> 00:31:05,200
you have one more question perfect then

791
00:31:05,200 --> 00:31:08,320
come to the microphone please

792
00:31:09,679 --> 00:31:13,120
i'm always a bit queer tinker

793
00:31:13,120 --> 00:31:15,760
have you you have a device with multiple

794
00:31:15,760 --> 00:31:18,559
cameras on it for

795
00:31:18,559 --> 00:31:19,760
have you ever

796
00:31:19,760 --> 00:31:22,080
done or built someone let's say a room

797
00:31:22,080 --> 00:31:24,880
with the cameras around and then put the

798
00:31:24,880 --> 00:31:27,919
360 software on

799
00:31:27,919 --> 00:31:29,919
yeah i know what you mean yeah actually

800
00:31:29,919 --> 00:31:32,399
egs supports that also

801
00:31:32,399 --> 00:31:33,760
basically it's

802
00:31:33,760 --> 00:31:35,600
reversing the direction you're looking

803
00:31:35,600 --> 00:31:38,399
no longer looking outside from from a

804
00:31:38,399 --> 00:31:41,120
sphere but you're looking onto something

805
00:31:41,120 --> 00:31:43,120
yes you can do that but i haven't played

806
00:31:43,120 --> 00:31:44,320
around with it but i saw that it's

807
00:31:44,320 --> 00:31:47,279
supported yes

808
00:31:47,279 --> 00:31:49,279
i now know let's support it so yeah

809
00:31:49,279 --> 00:31:51,120
maybe so if you want to play around with

810
00:31:51,120 --> 00:31:53,039
it yeah yeah

811
00:31:53,039 --> 00:31:56,399
and one more question

812
00:31:56,399 --> 00:31:58,080
have you seen um

813
00:31:58,080 --> 00:31:59,760
front ends for the end user to do the

814
00:31:59,760 --> 00:32:01,840
reframing thing so

815
00:32:01,840 --> 00:32:04,640
actually allowing the viewer to select a

816
00:32:04,640 --> 00:32:07,440
part of the the image or a moving target

817
00:32:07,440 --> 00:32:09,760
and keep following that

818
00:32:09,760 --> 00:32:12,000
um

819
00:32:12,000 --> 00:32:14,320
no automatic tracking in the front end i

820
00:32:14,320 --> 00:32:16,799
haven't seen that you know i mean i i've

821
00:32:16,799 --> 00:32:19,679
i've seen it in the insta360 cameras

822
00:32:19,679 --> 00:32:22,000
basically it's built into the into the

823
00:32:22,000 --> 00:32:22,880
app

824
00:32:22,880 --> 00:32:25,600
and it works pretty well

825
00:32:25,600 --> 00:32:28,399
but no that's actually

826
00:32:28,399 --> 00:32:30,480
would be a nice feature

827
00:32:30,480 --> 00:32:32,480
but also i think some some cameras

828
00:32:32,480 --> 00:32:34,559
support it in camera so

829
00:32:34,559 --> 00:32:36,240
if you

830
00:32:36,240 --> 00:32:37,760
if you want to do

831
00:32:37,760 --> 00:32:40,000
automatic re re-tracking anyway why

832
00:32:40,000 --> 00:32:41,679
don't you do it on the camera side and

833
00:32:41,679 --> 00:32:44,080
then you you're no longer dealing with a

834
00:32:44,080 --> 00:32:46,480
360 video which needs more bandwidth you

835
00:32:46,480 --> 00:32:48,720
know yeah the use case would be for

836
00:32:48,720 --> 00:32:50,480
instance for sports defense if you have

837
00:32:50,480 --> 00:32:53,039
your favorite player okay say i want to

838
00:32:53,039 --> 00:32:55,279
see this basketball player all game long

839
00:32:55,279 --> 00:32:58,080
yeah and not the rest but i can provide

840
00:32:58,080 --> 00:32:58,960
one

841
00:32:58,960 --> 00:33:01,600
360 video to everyone and they just pick

842
00:33:01,600 --> 00:33:03,600
whatever person they want to follow for

843
00:33:03,600 --> 00:33:05,120
them yeah that would be a nice

844
00:33:05,120 --> 00:33:06,080
i don't know

845
00:33:06,080 --> 00:33:08,399
premium feature or something

846
00:33:08,399 --> 00:33:10,080
okay and i see we have one question from

847
00:33:10,080 --> 00:33:12,960
the internet microphone yes exactly um

848
00:33:12,960 --> 00:33:16,240
the question was if you could

849
00:33:16,240 --> 00:33:20,159
go a little bit more into detail on

850
00:33:20,159 --> 00:33:23,360
bandwidth requirements

851
00:33:23,360 --> 00:33:24,799
yeah so

852
00:33:24,799 --> 00:33:27,360
basically um

853
00:33:27,360 --> 00:33:30,240
the resolution and the the frame rate

854
00:33:30,240 --> 00:33:32,880
are the most important factors

855
00:33:32,880 --> 00:33:36,360
and also uh you have on the one hand

856
00:33:36,360 --> 00:33:41,919
h.264 as a codec and or h.265 which is

857
00:33:41,919 --> 00:33:44,880
almost twice as efficient

858
00:33:44,880 --> 00:33:46,320
and

859
00:33:46,320 --> 00:33:47,120
the

860
00:33:47,120 --> 00:33:49,360
i would say the normal bandwidth

861
00:33:49,360 --> 00:33:52,320
estimates apply so if you have a 4k

862
00:33:52,320 --> 00:33:54,960
video stream

863
00:33:55,760 --> 00:33:58,240
at 30fps maybe

864
00:33:58,240 --> 00:34:01,120
you probably want ideally i mean ideally

865
00:34:01,120 --> 00:34:02,720
you want to have as much bandwidth as

866
00:34:02,720 --> 00:34:03,840
you can so

867
00:34:03,840 --> 00:34:05,840
this camera supports 40 megabits for

868
00:34:05,840 --> 00:34:08,960
stitched or 120 megabits for unstitched

869
00:34:08,960 --> 00:34:10,079
images

870
00:34:10,079 --> 00:34:11,839
so if you want to get the best quality

871
00:34:11,839 --> 00:34:15,279
you would probably stitch it

872
00:34:15,760 --> 00:34:19,879
on the server um

873
00:34:20,239 --> 00:34:21,520
but

874
00:34:21,520 --> 00:34:23,760
you can also get away with

875
00:34:23,760 --> 00:34:25,679
20 megabits i think that's what i'm

876
00:34:25,679 --> 00:34:28,159
using right now and it's for 3d i mean

877
00:34:28,159 --> 00:34:30,399
it doesn't look really great

878
00:34:30,399 --> 00:34:31,760
i was testing it during the camp the

879
00:34:31,760 --> 00:34:35,119
whole time with wi-fi and it dropped out

880
00:34:35,119 --> 00:34:36,879
a lot of the time

881
00:34:36,879 --> 00:34:39,760
and then now i have ethernet here so i

882
00:34:39,760 --> 00:34:42,879
can crank it up a bit

883
00:34:45,359 --> 00:34:46,839
thanks a lot

884
00:34:46,839 --> 00:34:49,520
okay and if there are no more questions

885
00:34:49,520 --> 00:34:54,119
thank you very much for your talk thanks

886
00:35:03,280 --> 00:35:05,359
you